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The  U.S.  Air  Force  operates  and  maintains  some  of  the 
most  sophisticated  and  complex  aerospace  vehicles  in  the 
world.  Technicians  maintaining  its  most  advanced  aircraft, 
such  as  the  F-16,  B-2,  and  F-22,  are  expected  to  tise 
technical  data  displayed  on  electronic  media  rather  than 
conventional  paper  Technical  Orders  (T.O.s).  Recent 
advances  in  display  technology  and  computer  component 
miniaturization  have  made  it  possible  to  produce  fully 
portable  monocular  head-mounted  display  systems  as  well  as 
standard  flat-screen  displays.  As  both  types  of  electronic 
display  devices  offer  a  desirable  means  for  presenting 
aircraft  maintenance  technical  data  to  technicians,  this 
research  attempted  to  determine  if  flightline  maintenance 
performance  depends  on  the  display-type  used. 
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Abstract 

As  military  developers  provide  increasingly  complex 
weapon  systems,  it  becomes  more  difficult  for  maintenance 
technicians  to  perform  their  jobs.  One  aspect  of  the 
technicians'  world  is  the  need  to  access  technical 
information  in  the  performance  of  their  duties.  This  study 
investigated  two  electronic  display  systems  to  evaluate 
which  enhanced  technician  performance  more.  A  Head  Mounted 
Display  (HMD)  device  and  a  portable  hand-held 
flat-screen  computer  were  evaluated  in  the  performance  of 
two  flightline  maintenance  activities.  Although  both 
display  systems  were  fully  portable  and  self  contained, 
only  the  HMD  system  allowed  continuous  access  to  technical 
information  during  task  performance.  In  most  cases, 
technicians  using  the  HMD  system  outperformed  those  equipped 
with  the  flat-screen  computer  system  in  terms  of 
effectiveness  and  efficiency. 
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COMPARATIVE  EVALUATION  OF  A  MONOCULAR  HEAD  MOUNTED 


DISPLAY  DEVICE  VERSUS  A  FLAT  SCREEN  DISPLAY  DEVICE  IN 
PRESENTING  AIRCRAFT  MAINTENANCE  TECHNICAL  DATA 


I.  Introduction 


The  Need  for  Research 

Today,  computer  application  covers  nearly  all  aspects 
of  society.  Methods  of  displaying  information  generated  by 
these  computers  are  as  diverse  as  the  applications 
themselves.  Typical  displays  used  on  Personnel  Computers 
are  the  desktop  monitor  and  the  flip-up  display  screens  used 
on  portable  laptop  computers.  An  alternative  method  of 
displaying  information  is  the  monocular  head-mounted  display 
device  (HMDD) .  The  HMDD  has  been  used  primarily  in  military 
aviation.  For  example,  Army  pilots  flying  AH-64  Apache 
helicopters  use  HMDD  technology  for  displaying  target  and 
flight  information.  An  HMDD  is  a  miniaturized  display  that 
presents  an  image  to  one  of  the  wearer's  eyes.  Although  the 
HMDD  screen  may  be  physically  as  small  as  a  one  inch  by  one 
inch  square,  its  image  is  equivalent  to  that  presented  by  a 
12-inch  computer  monitor  viewed  at  two  feet  (Masquelier, 
1991:2).  A  key  attribute  of  HMDD  technology  is  that  it 
permits  ready  accessibility  to  information  while  allowing 
both  hands  to  be  free  to  perform  an  activity.  HMDD 
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technology  has  proven  its  worth  in  the  cockpit;  expansion  of 
HMDD  technology  into  non-flying  applications  appears  to  have 
merit. 

Performance  Enhancement  for  Aircraft  Maintenance. 
Civilian  aircraft  maintenance  has  become  an  increasingly 
difficult  activity.  Worldwide  accidents  and  incidents 
attributed  to  improper  "inspection,  servicing,  and  repair  of 
aircraft"  are  on  the  rise  (Shepherd,  1990:1168). 

Maintenance  concerns  within  the  Federal  Aviation 
Administration  (FAA)  prompted  "a  multiyear  research  program 
to  identify  areas  where  human  factors  engineering  can 
significantly  enhance  safety...."  (Shepherd,  1990:1167). 

One  of  the  FAA's  primary  research  areas  is  the  application 
of  advanced  technology  to  aircraft  mechanic  performance. 

The  increasing  complexity  of  today's  aircraft  stresses  the 
technician's  ability  to  maintain  the  equipment.  For 
example,  some  aircraft  are  so  sensitive  that  "misapplication 
of  voltmeter  leads  can  cause  thousands  of  dollars  of  damage" 
(Shepherd,  1990:1168).  The  FAA  believes  that  "humans  will 
not  be  able  to  meet  the  growing  challenges  of  aircraft 
maintenance  without  the  proper  application  of  technology" 
(Department  of  Transportation,  1991:105).  To  compound  the 
problem  of  increased  aircraft  complexity,  the  talent  pool 
from  which  the  airlines  and  the  military  can  recruit 
technicians  is  shrinking  (Shepherd,  1990:1169).  Therefore, 
engineers  and  scientists  involved  in  human  factors  research 
should  seek  to  improve  the  performance  of  technicians  that 
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remain  in  the  career  field  (Shepherd,  1990:1169). 

Becker  states  that  "ready  accessibility  to  hands-free, 
superimposed  information  could  be  invaluable"  in 
applications  that  relate  to  patient  care  in  medical  settings 
and  to  the  assembly  and  maintenance  of  high  cost  computers 
and  engines  (Becker,  1990:10).  The  civilian  aircraft  sector 
has  started  to  use  HMDDs  in  the  ground  based  aerospace 
environment  for  the  same  reason.  For  instance,  one  company 
is  developing  an  HMDD  system  for  aerospace  mechanics.  The 
system  allows  the  mechanic  to  retrieve  instructions  or 
graphics  while  performing  aircraft  assembly  activities 
(Schwind,  1990:26). 

By  using  computer  systems  to  display  technical  data, 
technicians  would  be  better  able  to  keep  pace  with  the 
demands  of  complex  aircraft.  The  FAA  wishes  to  improve  upon 
the  current  use  of  "paper"  technical  information  by 
implementing  "computer-based"  methods  (Shepherd,  1990:1170). 
Hand-held  portable  computers  and  HMDDs  could  be  part  of 
these  preferred  computer-based  methods. 

Armstrong  Laboratory  Mission.  The  F-16,  B-2,  and  F-22 
are  but  a  few  of  the  highly  complex  aircraft  operated  and 
maintained  by  the  US  Air  Force  (USAF) .  The  technicians 
maintaining  these  aircraft  are  expected  to  use  technical 
data  displayed  on  electronic  media  (computers  with  flat 
screens)  rather  than  conventional  paper  Technical  Orders 
(T.O.s).  In  addition  to  developing  the  technology  needed  to 
display  T.O.s  electronically,  Armstrong  Laboratory  (AL)  has 
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been  tasked  by  the  USAF  to  determine  the  degree  to  which 
alternative  methods  of  presentation  enhance  maintenance 
technician  performance. 

Integrated  Maintenance  Information  System.  AL 

established  the  Integrated  Maintenance  Information  System 

(IMIS)  effort  to  determine  if  more  accessible  and  accurate 

technical  data  would  improve  maintenance  technician 

performance.  In  developing  a  flat  screen  portable  computer, 

the  IMIS  study  determined  a  limitation. 

The  currently  envisioned  computer  will  not  permit  the 
technician  to  have  technical  data  available  at  all 
times  and  will  not  physically  fit  in  all  compartments 
on  the  aircraft.  (Thomas,  1990) 

As  part  of  the  IMIS  effort,  AL  sponsored  a  study  performed 

by  Barbara  Masguelier  to  investigate  an  alternative  display 

that  would  alleviate  this  limitation.  Masguelier 's  study 

compared  the  use  of  a  monocular  HMDD  to  a  portable  laptop 

computer,  referred  to  as  a  Grid  computer,  in  an  avionics 

maintenance,  bench-level,  repair  shop  environment. 

Masguelier 's  research  objective  was  to  determine  the 

extent  of  any  performance  differences  between  technicians 

using  the  two  display  devices.  She  found  no  statistically 

significant  advantages  to  using  one  device  over  the  other  in 

a  bench-level  maintenance  task  (Masguelier,  1991:67). 

However,  her  study  also  recommended  that  the  two  display 

devices  should  be  tested  in  a  flight line  maintenance 

environment  because  it  would  provide  a  more  appropriate  test 

for  the  HMDD  (Masguelier,  1991:66).  The  HMDD  was  designed 
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to  permit  a  technician  to  have  both  hands  free  to  manipulate 
the  equipment  being  worked  on.  It  also  provides  the 
technician  with  less  restricted  mobility  to  perform  the  task 
while  the  technical  data  is  continuously  displayed. 

Research  Focus 

The  previous  discussion  suggested  there  was  a  need  for 
a  comparative  evaluation  of  display  devices  being  used  in 
the  performance  of  a  flight line  task.  The  current  research 
was  performed  to  determine  if  technician  performance 
improves  when  aircraft  maintenance  technical  data  is 
presented  on  an  HMOD  as  opposed  to  a  hand-held,  flat-screen 
device,  referred  to  a  Portable  Maintenance  Aid  (PMA) .  To 
quantify  the  concept  of  performance,  the  researchers  chose 
to  measure  task  completion  times  and  the  number  of  faults 
detected.  PMA  placement  was  specifically  selected  to 
preclude  the  technician  from  having  continuous  access  to 
technical  data  while  performing  the  maintenance  activity. 

Scope  and  Limitations 

The  HMDD  used  in  the  current  research  was  a 
commercially  available  Cathode  Ray  Tube  (CRT)  display 
manufactured  by  Imaging  &  Sensing  Technology.  All  the  other 
components  of  the  HMDD  system  were  provided  by  AL.  These 
components  were  mounted  in  a  lightweight  vest,  resulting  in 
a  fully  portable  ensemble  suitable  for  flight line 
applications.  A  more  detailed  description  of  the  HMDD 
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system  can  be  found  in  Appendix  A.  Another  HMDD  display, 
the  Private  Eye®,  was  available  but  not  used  in  this  study; 
its  Light  Emitting  Diode  (LED)  screen  resolution  did  not 
meet  IMIS  requirements.  The  PMA  used  in  this  study  was  a 
flat-screen  computer  that  had  input  keys  surrounding  a  six 
inch  by  eight  inch  Liquid  Crystal  Display  screen.  The  PMA's 
physical  characteristics  can  be  found  Appendix  A. 

The  tasks  selected  for  this  study  were  limited  by  two 
factors:  the  impact  of  testing  on  the' host  organization  and 
the  short  battery  life  of  the  prototype  HMDD  system.  The 
tasks  selected  were  limited  to  less  than  30  minutes  per 
trial  to  accommodate  the  host  organization's  need  to 
minimize  disruption  of  normal  operational  duties. 
Additionally,  the  HMDD  system  ran  on  two  rechargeable 
batteries,  which  would  be  completely  drained  after  30 
minutes  of  operation.  AL  stated  that  battery  life  on 
production  HMDD  systems  would  be  improved  over  the  prototype 
used  for  this  research.  Moreover,  the  combination  of  an 
insufficient  number  of  spare  batteries  and  a  long  charging 
cycle  prevented  changing  batteries  while  conducting  a  trial. 

When  evaluating  electronic  media  displays,  large  data 
bases  of  technical  data  that  require  logical  branching  to 
perform  the  task  should  be  used  (Nugent  and  others, 

1987:12).  However,  the  tasks  with  a  high  degree  of 
complexity  and  information  branching  available  for  this 
study  would  have  exceeded  30  minutes  and  so  could  not  be 
used. 
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The  location  of  the  current  research  was  the  178th 
Tactical  Fighter  Group  (TFG) ,  Springfield,  Ohio,  Air 
National  Guard  (OANG) .  The  test  article  was  the  A-7D 
Corsair  II  attack  aircraft  assigned  to  the  unit.  Two  tasks 
were  selected  for  testing  during  the  current  experiment. 

One  task  was  performed  with  the  test  subject  seated  in  the 
cockpit  of  the  aircraft  and  the  other  was  performed  with  the 
technician  inspecting  the  aircraft  engine  bay.  The 
maintenance  activities  available  for  test  were  limited  to 
the  systems  on  this  aircraft. 

SWMyaCY 

The  sections  above  discuss  how  computers  are  becoming 
more  prevalent  in  the  civilian  and  military  sectors.  They 
also  suggest  that  the  scope  of  potential  applications  is 
widening  due  to  the  need  for  keeping  pace  with  the  ever 
increasing  complexity  of  today’s  systems.  Both  the  FAA  and 
the  Air  Force  feel  that  aircraft  maintenance  specialists 
will  need  computers  to  aid  in  maintaining  the  complex 
aircraft  of  the  future.  This  research  was  conceived  to 
compare  promising  computer  display  technologies  for  such  an 
application.  Various  other  factors  shaping  this  experiment 
were  also  outlined.  In  the  following  chapter,  a 
presentation  of  past  computer  research  dealing  with  aircraft 
maintenance  and  its  relation  to  this  study  is  provided. 
Additionally,  a  discussion  of  the  interaction  between  man 
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and  computer  and  the  associated  advantages  and  disadvantages 
of  computer  use  are  presented. 


8 


II.  Background 


Qygrview 

The  relationship  between  the  hiunan-coxuputer  interface 
and  performance  potential  in  a  maintenance  environment  is  a 
complex  and  largely  untested  field  of  study.  This  chapter 
will  review  four  areas  that  influenced  the  formulation  of 
the  current  experiment:  (1)  previous  HMDD  nonflying 
experimentation,  (2)  user  performance  potential,  (3)  visual 
side  effects,  and  (4)  display  attributes. 

Previous  HMDD  Non-Flvina  Experimentation 

To  the  knowledge  of  the  current  researchers,  only  two 
studies  have  been  performed  that  evaluated  the  application 
of  HMDD  technology  to  non-f lying  activities.  Both  studies 
compared  HMDD  systems  to  flat~screen  computer  systems  in  an 
aircraft  maintenance  environment.  Studies  of  HMDD 
applications  in  flying  environments  were  investigated  and 
determined  to  be  inappropriate  due  to  the  vast  differences 
in  operating  environments.  This  study  concentrated  on  the 
impact  of  HMDD  application  to  activities  in  a  comparatively 
static  environment. 

Masquelier  Study.  The  first  study  to  be  discussed  was 
performed  by  Barbara  Masquelier  in  1991.  Masquelier 
conducted  a  controlled  experiment  of  two  similar  bench-top, 
trouble-shooting  tasks  using  a  Light  Emitting  Diode  (LED) 
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HMDD  and  a  flat-screen  device  (Grid  Computer)  to  display  the 
technical  information.  The  experiment  collected  two  types 
of  data:  (1)  observational  measurements  from  the  experiment 
itself  and  (2)  responses  from  post-test  questionnaires  and 
structured  interviews.  The  participants  were  16  maintenance 
technicians  (eight  experienced  and  eight  less-experienced) 
from  the  4950th  Test  Wing  Communication  and  Navigation  shop 
(Masquelier,  1991:7). 

Masquelier  found  no  statistically  significant 
performance  difference  between  HMDD  equipped  technicians  and 
Grid  Computer  equipped  technicians  (Masquelier,  1991:67). 
However,  her  study  provided  some  key  elements  to  the  current 
experimental  design:  (1)  a  recommendation  for  testing 
flightline  activities,  (2)  identification  of  technician 
experience  as  a  possible  influence  on  task  completion  times, 
and  (3)  an  assessment  of  HMDD  visual  effects  on  subjects 
performing  in  a  relatively  static  environment. 

Testing  Fliahtline  Activities.  The  task  performed 
in  Masquelier 's  study  was  a  bench-top  checkout  and  analysis 
of  a  radio  receiver/ transmitter.  This  type  of  checkout  was 
considered  by  the  4950th  Test  Wing  to  be  representative  of 
the  intermediate  level  maintenance  actions  it  routinely 
performs  (Masquelier,  1991:35).  However,  the  task  did  not 
require  the  technician  to  move  from  the  workbench  to  perform 
the  checkout,  nor  did  it  require  both  hands  to  be  free. 
Masquelier  concluded  that  such  a  task  was  not  a  suitable 
choice  to  test  a  hands-free  device  that  offered  the 
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technician  a  high  degree  of  nobility  (Masquelier,  1991:67). 
Masquelier's  assessnent  was  echoed  by  technicians  perfoming 
the  troubleshooting  task.  All  the  technicians  agreed  that 
"the  tasks  selected  were  not  the  best  possible  application 
for  the  use  of  a  hands-free  display  device"  (Masquelier, 
1991:66).  Therefore,  further  experimentation  involving  a 
flightline  maintenance  task  was  recommended  by  Masquelier  to 
better  test  the  HMDD  against  the  portable  Grid  computer 
(Masquelier,  1991:66). 

Technician  Experience.  Masquelier's  study  found 
that  "there  was  a  statistically  significant  difference  for 
the  performance  times  of  experienced  and  [less-experienced] 
technicians"  (Masquelier,  1991:69).  The  degree  of  influence 
experience  has  on  technician  performance  will  be  discussed 
later  in  the  User  Performance  Differential  section. 

HMDD  visual  Side  Effects.  Masquelier's 
questionnaire  and  interview  data  suggested  that  visual 
problems  such  as  eye  strain,  blurring,  focusing,  capability 
to  switch  attention  from  the  display  to  the  work,  headaches, 
and  afterimages  were  not  a  problem  for  technicians 
performing  the  bench-level  test  with  the  HMDD  (Masijuelier, 
1991:73).  Moreover,  the  technicians  found  both  devices 
suitable  for  performing  maintenance  activities,  although 
they  preferred  to  use  the  Grid  computer  (Masquelier, 
1991:70).  Additional  discussion  of  visual  side  effects  is 
provided  after  a  review  of  the  Edwards  study. 
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Edwards  Study.  The  second  evaluation  of  display 
devices  was  a  comparative  analysis  of  HMDD  and  flat-screen 
technology  performed  by  General  Dynamics  Electronics 
Division  in  November  1990.  The  study  was  based  on  the 
personal  perceptions  of  aircraft  maintenance  technicians 
assigned  to  the  F-16  Combined  Test  Force  organization.  The 
study  asked  technicians  to  compare  LED  and  CRT  HMDDs  to 
hand-held  computers.  Technicians  "completed  a  rating  scale 
and  commented  on  open-ended  questions"  (Edwards  Evaluation 
Report,  1991:16).  Their  responses  were  combined  and  a 
statistical  mean  was  calculated  for  each  of  the  question 
categories. 

The  Edwards  study  provided  the  following  key  aspects  to 
the  design  of  the  current  experiment:  (1)  a  measure  of 
display-types'  suitability  to  assist  the  technician  in  a 
maintenance  activity  (2)  a  confirmation  of  the  importance 
of  task  selection,  and  (3)  an  assessment  of  the  visual  side 
effects. 

Display  Type  Suitability.  The  overall  mean  rating 
indicated  that  both  types  of  devices  were  found  acceptable 
for  displaying  technical  data  in  an  environment  which  was 
not  highly  dynamic.  When  the  study's  20  technicians  were 
asked  which  display  device  they  preferred  to  use,  18 
preferred  the  flat-screen  devices,  and  two  preferred  the 
HMDDs  (Edwards  Evaluation  Report,  1991:26).  The  perceptions 
of  the  participants  of  the  Edward's  study  confirm  what  the 
Masquelier  study  concluded;  technicians  prefer  to  use  a 
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flat-screen  display  device  over  the  use  of  an  HMDD. 

However,  both  studies  provided  very  little  time  for 
technicians  to  familiarize  themselves  with  the  visual 
demands  associated  with  using  an  HMDD. 

Importance  of  Task  Selection.  When  the  20 
technicians  were  asked  whether  they  would  prefer  using  the 
flat-screen  device  (or  PMA)  or  the  HMDD  under  different 
circumstances,  ten  preferred  to  use  the  PMA  under  all 
conditions.  None  of  the  technicians  preferred  to  use  the 
HMDD  under  all  conditions,  but  ten  preferred  to  have  both 
systems  available  for  all  applications  (Edwards  Evaluation 
Report,  1991:26).  This  preference  for  both  display  systems 
suggests  that  the  selection  of  a  display  device  might  be 
task  dependent.  Further  evidence  of  task  dependency  was 
seen  in  comments  made  by  the  technicians.  There  were  three 
primary  reasons  given  for  preferring  the  flat-screen 
devices.  The  major  reason  given  was  the  freedom  provided  by 
not  having  anything  attached  to  the  body;  a  technician  can 
lay  it  aside  when  not  immediately  using  it.  One  technician 
commented  that  "accessing  tight  places  will  make  wearing 
(the  HMDD)  impractical"  (Edwards  Evaluation  Report, 

1991:27).  A  second  reason  given  was  the  compactness  of  the 
flat-screen  devices.  The  third  reason  was  its  ease  of  use. 
In  contrast,  reasons  given  for  preferring  the  HMDD  were 
characteristic  of  maintenance  activities  that  differ  from 
activities  in  which  flat-screen  systems  excel.  For  example, 
one  reason  given  was  the  HMDD  would  allow  the  technician  to 
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have  both  hands  free  for  carrying  items  other  than  the 
technical  data  display  (Edwards  Evaluation  Report,  1991:26). 
Another  technician  suggested  that  the  HMDD  "might  be  good 
for  presenting  workcards  for  phase  inspection"  (Edwards 
Evaluation  Report,  1991:27).  Apparently,  technician  display 
preference  is  very  much  task  dependent. 

HMDD  Visual  Side  Effects.  Some  of  the 
participants  indicated  that  they  suffered  some  visual  side 
effects  from  using  the  HMDD.  The  study  placed  the  visual 
problems  into  three  categories:  eye  strain,  frustration  with 
trying  to  focus  on  information,  and  interference  between  eye 
piece  and  work  objects  (Edwards  Evaluation  Report,  1991:23). 
The  study  failed  to  mention  the  number  of  complaints 
associated  with  the  categories.  However,  the  descriptive 
words  used  indicated  that  the  participants  affected  were  in 
the  minority. 

User  Performance  Differential 

The  current  study  focused  on  factors  that  influence 

technician  performance.  The  interaction  between  man  and 

machine  appears  to  affect  performance  differently  depending 

on  the  experience  of  the  individual  and  the  type  of  task 

performed.  According  to  Heleander: 

Users  who  have  acquired  extensive  knowledge  and  skill 
related  to  a  job  might  be  expected  to  use  a  computer 
system  on  the  job  more  effectively  than  users  with 
little  domain  specific  knowledge.  (Heleander,  1988:557) 

Heleander 's  view  appears  to  be  confirmed  in  Masquelier's 
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study;  experienced  technicians  using  the  display  devices 
outperformed  their  less-experienced  counterparts. 

The  Edwards  study  suggested  that  the  performance  of 
inspection  tasks  might  benefit  from  using  an  HMDD,  while 
Drury  and  others  (1989)  implied  that  inspection  performance 
is  a  function  of  cognition.  Taken  together,  these  studies 
could  be  interpreted  to  infer  that  cognitive  load  influences 
the  degree  to  which  electronic  display  devices  enhance 
technician  performance.  Hence,  cognitive  load  was 
considered  in  selecting  the  tasks  evaluated  in  the  current 
study. 

HMDD  Visual  Side  Effects 

Both  the  Masquelier  and  Edwards  study  dealt  with 
possible  visual  problems  that  result  from  using  an  HMDD  in  a 
maintenance  environment.  The  concern  over  visual  problems 
stems  from  studies  evaluating  HMDD  technology  in  a  flying 
environment.  A  study  by  Rash  and  Martin  suggested  retinal 
rivalry  and  a  slight  decrease  in  visual  resolution  might 
result  from  using  an  HMDD  in  a  dynamic  environment.  Retinal 
rivalry  is  the  ability  to  "selectively  switch  back  and  forth 
between  the  two  images  being  presented  to  separate  eyes" 
(Rash  and  Martin,  1988:M-3-9).  A  Hale  and  Piccoione  study 
supposed  that  depth  perception  difficulties  (spatial 
disorientation)  might  occur  from  using  an  HMDD  (Hale  and 
Piccione,  1990:14).  The  current  experimental  environment 
falls  between  the  relatively  stable  bench-top  environment  of 
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Masguelier's  study  and  the  highly  dynamic  environment  of  a 
pilot  flying  a  plane.  The  current  study  will  investigate 
these  potential  effects  on  a  technician  performing 
flightline  tasks. 

The  side  effects  experienced  by  participants  in  the 
studies  discussed  thus  far  may  have  been  a  function  of 
environmental  ergonomic  factors.  With  the  help  of  the 
American  Optometric  Association  (AOA) ,  Sheedy  collected  data 
and  ranked  the  eight  most  common  types  of  problems 
associated  with  working  at  a  Visual  Display  Terminal.  The 
problems  were  as  follows:  (1)  eyestrain,  (2)  headaches,  (3) 
blurred  Vision,  (4)  dry  or  irritated  eyes,  (5)  neck  and/or 
backaches,  (6)  photophobia,  (7)  double  vision,  and  (8) 
afterimages  (Sheedy,  1992:20).  Of  the  total  number  of 
responses  in  the  Sheedy  study,  37%  of  the  subjects  were 
diagnosed  as  having  problems  attributable  to  environmental 
factors.  The  optometrists  of  the  AOA  ranked  four 
environmental  ergonomic  factors  as  the  cause  of  the 
problems:  (1)  screen  glare,  (2)  work  arrangement,  (3)  poor 
lighting,  and  (4)  screen  resolution  (Sheedy,  1992:20-21). 

The  environment  of  the  current  study  addresses  these 
ergonomic  factors. 

Pigplay 

with  regard  to  the  current  investigation,  the  interface 
between  man  and  machine  begins  with  the  display  screen,  or 
more  specifically,  the  attributes  of  the  information 
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displayed  on  that  screen.  A  n\imber  of  studies  have 
suggested  that  human  performance  degrades  as  the  density  of 
information  displayed  on  one  computer  screen  increases 
(Heleander,  1988:382).  When  programmers  are  deciding  what 
information  to  display,  they  should  display  only  the  minimiun 
amount  of  information  necessary  to  complete  the  task. 
However,  there  is  no  established  density  threshold  which 
should  not  be  crossed  (Heleander,  1988:387).  Text 
arrangement  is  also  a  concern.  Galitz  (1985)  states: 
"Reserve  specific  areas  of  the  screen  for  certain  kinds  of 
information,  such  as  commands,  error  messages,  and  input 
fields,  and  maintain  these  areas  consistently  on  all 
screens. " 

Presentation,  as  well  as  arrangement,  of  the  text  is 
also  a  consideration.  Both  upper  and  lower  case  letters 
should  be  used,  as  in  conventional  writing.  Text  displayed 
in  this  manner  is  read  about  13%  quicker  than  text  in  all 
upper  case  (Heleander,  1988:387).  Left  margin  justification 
and  double  line  spacing  should  be  used.  Single  spacing  is 
read  11%  slower  than  double  spacing.  Double  spacing  should 
be  used  between  paragraphs  also  (Heleander,  1988:398-399). 
Even  if  the  text  is  arranged  as  above,  a  poor  cpiality  image 
will  degrade  its  useability.  To  facilitate  a  proper  image, 
the  use  of  a  monochrome  screen  is  preferred.  This  screen 
permits  adequate  contrast  between  the  text  and  the 
background  (Heleander,  1988:437,439). 
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In  the  current  research,  the  software  (image)  provided 
for  both  the  HMDD  and  PMA  displayed  text  left  justified  with 
upper  and  lower  case  letters.  The  text  used  single  line 
spacing  with  double  line  spacing  between  steps.  Both 
devices  used  a  monochrome  screen  with  black  lettering  on  a 
white  background.  The  text  was  consistently  placed  in  a 
specific  region  of  the  screen  with  error  messages,  command 
lines,  and  input  fields  placed  in  separate  areas  of  the 
screen.  All  of  these  display  attributes,  except  for  the 
single  line  spacing,  should  further  contribute  to 
performance  enhancement.  Line  spacing  was  constrained  by 
the  software  system  provided  by  AL. 

PMA  Screen  Attributes.  The  screens  of  large, 
battery-powered,  flat-screen  LCDs  have  historically  had 
contrast  limitation  problems,  resulting  in  difficulties  with 
certain  viewing  angles.  However,  recently  developed  LCDs 
don't  necessarily  exhibit  this  problem  to  as  great  an  extent 
(Heleander,  1988:470).  The  current  study  included  an 
evaluation  the  PMA's  contrast  capabilities. 

HMDD  Screen  Attributes.  Historically,  KMDDs  have  had 
problems  with  image  quality.  The  high  resolution  of  a  Video 
Graphic  Array  (VGA)  CRT  makes  it  a  suitable  choice  for 
displaying  pictorial  information,  alphanumerics  and  graphics 
(Tannas,  1985:138).  It  appears  plausible  that  HMDD  image 
quality  may  be  related  to  users'  susceptibility  to  eye 
strain  with  subsequent  headaches  and  blurred  vision. 

Improved  image  quality  enhances  the  opportunity  for 


18 


increased  technician  performance  due  to  the  VGA  CRT 
capability  to  display  detailed  graphics  and  high  quality 
text. 

Screen  fading  is  another  disadvantage  associated  with 
electronic  displays.  Controlling  target  luminance  is  a 
common  method  of  reducing  screen  fading.  However,  the 
display  liminance  must  be  set  to  avoid  causing  user  health 
problems.  In  a  study  performed  by  Matthews  and  Mertins 
(1987),  target  luminance  was  tested  at  various  levels 
normally  found  on  microcomputer  display  screens.  The  test 
period  was  four  hours,  the  length  of  time  thought  equal  to 
half  a  work  day.  The  study  concluded  that  neither  visual 
performance,  nor  subjective  well-being,  were  significantly 
influenced  by  the  screen  luminance  (Matthews  and  Mertins, 
1987:1275).  The  screen  luminance  of  the  HMDD  used  in  the 
current  study  was  below  the  luminance  level  cited  by 
Matthews  and  Mertins. 

Summary 

The  research  discussed  above  highlights  important  items 
for  this  study  to  consider,  both  from  an  evaluation  and  an 
experimental  controls  perspective.  Factors  that  would 
appear  to  influence  performance  enhancement  include:  type  of 
task,  technician  experience  level,  and  display  attributes. 
Visual  side  effects  also  play  a  major  role  in  the  acceptance 
of  the  HMDD  as  a  performance  enhancement  tool.  In  the 
methodology  chapter  to  follow,  the  experimental  setup. 
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experimental  controls,  and  method  of  measuring  technician 
performance  are  addressed. 
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III.  Methodology 


Research  Design  Strategies 

In  this  study,  the  relationship  between  technician 
performance  and  the  method  used  to  display  technical  data 
was  researched.  To  facilitate  the  research,  data  had  to  be 
collected  and  evaluated.  Several  methods  of  collecting  and 
evaluating  data  were  available.  However,  constraints 
quickly  diminished  the  research  designs  suitable  for  this 
study.  The  primary  constraint  to  using  simulation  or  a 
parametric  study  was  the  limited  degree  of  problem 
crystallization:  the  research  question  lacked  sufficient 
definition  to  provide  the  structure  required  for  either  of 
these  methods.  Furthermore,  case  studies  and  surveys  were 
eliminated  because  the  AL-developed  HMDD  used  was  the  only 
fully  portable  HMDD  system  in  the  Air  Force  inventory  at  the 
time  of  this  study. 

A  true  experimental  design  was  selected  and  collection 
of  empirical  data  was  accomplished  by  direct  observation  of 
subjects  performing  maintenance  tasks  in  a  controlled 
experimental  environment.  The  artificiality  of  a  controlled 
experiment  was  minimized  by  conducting  the  trials  using 
operational  technicians  performing  standard  maintenance 
activities.  The  technical  information  displayed  was  similar 
to  the  T.O.  information  routinely  used  by  the  technicians. 
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A  previous  study  conducted  by  Masquelier  (1991)  used  a 
one-variable^at-a-time  empirical  methodology  to  investigate 
the  completion  times  of  bench-top  maintenance  activities. 
That  study  evaluated  the  influence  both  technician 
experience  level  and  display  type  had  in  effecting  task 
completion  time.  Masquelier 's  study  evaluated  16  subjects 
(eight  were  experienced  and  eight  were  less-experienced)  as 
they  each  performed  two  similar  tasks  for  a  total  of  32 
trials  (Masquelier,  1991:31).  This  method  was  not  selected 
for  the  current  study  because  it  restricted  the  range  of 
maintenance  activities  that  could  be  investigated.  The  two 
tasks  chosen  in  Masquelier 's  study  had  to  be  essentially 
equivalent  to  ensure  that  the  tasks  performed  had  an 
insignificant  influence  on  the  task  completion  times. 
Furthermore,  her  study  was  limited  to  two  tasks  in  order  to 
minimize  the  effect  of  technician  learning  on  the  completion 
times.  The  current  researchers  determined  that  using  the 
one-variable-at-a-time  method  would  have  required  reducing 
the  scope  of  the  current  investigation. 

A  factorial  design  was  chosen  for  this  study  because  it 
allowed  simultaneous  manipulation  of  several  treatments 
(factors) .  "By  a  factorial  experiment,  we  mean  that  in  each 
complete  trial  or  replication  of  the  experiment,  all 
possible  combinations  of  the  level  of  the  factors  are 
investigated"  (Montgomery,  1976:121).  Factorial  experiments 
are  the  most  efficient  designs  for  effects  analysis  when 
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there  is  more  than  one  factor  being  investigated 
(Montgomery,  1976:121).  Factorial  designs  are  commonly  used 
to  screen  possible  factors  when  research  is  in  the 
exploratory  stage.  For  the  current  study,  a  two-level, 
full-factorial  design  was  selected.  The  experiment 
investigated  three  independent  variables  (factors)  with  two 
levels  for  each.  This  design  is  called  a  2  X  2  X  2  or  2^ 
design,  which  provides  the  smallest  number  of  treatment 
combinations  with  which  three  factors  can  be  studied  in  a 
complete  factorial  arrangement.  (Montgomery  1976:180-181) 

An  additional  benefit  of  factorial-designed  experiments 
is  the  ability  to  evaluate  the  possible  interactions  each 
variable  has  with  regard  to  the  other  variables  in  the 
design.  By  contrast,  a  limitation  of  the  one-variable-at-a- 
time  approach  is  that  each  experimental  variable  is  assumed 
to  be  independent  of  the  other  input  variables.  "The  one- 
variable-at-a-time  method  provides  an  estimate  of  the  effect 
of  a  single  variable  at  selected  fixed  conditions  of  the 
other  variables  (Box,  1978:312)."  By  using  a  factorial 
design,  the  assumption  of  variable  independence  can  be 
empirically  assessed. 

In  summary,  a  two-level  factorial  design  was  selected 
for  studying  the  relationship  between  technician  performance 
and  the  method  used  to  display  technical  data  for  the 
following  reasons:  (1)  fewer  trials  are  required;  (2)  more 
factors  are  explored;  (3)  there  are  less  restrictive 
ass\imptions  regarding  independence  and  (4)  factorial  design 
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is  more  efficient  at  screening  the  influence  of  several 
variables  on  the  dependent  variable. 

Experimental  Design 

The  first  step  in  defining  the  experimental  design  was 
to  determine  the  responses  to  be  measured.  The  research 
question  considers  the  impact  of  several  factors  on 
technician  performance.  Thus,  this  study  selected  two 
measures  to  assess  technician  performance.  The  first 
measure,  task  completion  time,  relates  to  performance 
efficiency.  The  second  measure,  number  of  faults  found, 
relates  to  performance  effectiveness.  To  obtain  a  measure 
of  the  participants'  attitudes  and  perceptions,  each 
technician  was  surveyed  and  subsequently  interviewed  to 
ascertain  their  opinions  on  the  use  of  both  the  PMA  and 
HMDD.  The  survey  and  interview  questions  are  included  in 
Appendices  B  and  C,  respectively.  Additional  data,  obtained 
to  supplement  the  quantitative  and  qualitative  results, 
included  a  background  questionnaire  (see  Appendix  D) ,  a 
visual  resolution  test,  and  an  eye  dominance  exam.  The 
background  questionnaire  obtained  information  on  previous 
job  experience,  while  the  visual  resolution  test  measured 
the  corrected  far  vision  of  the  participants.  Technicians 
wearing  glasses  were  tested  with  their  glasses  on  since  they 
would  be  wearing  them  during  the  experiment. 

The  second  step  in  defining  the  experimental  design  was 
to  select  the  experimental  factors  and  their  respective 
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levels.  A  factor  is  defined  as  the  characteristic  that 
differentiates  the  treatments  or  populations  from  one 
another.  The  different  treatments  or  populations  are 
referred  to  as  the  levels  of  the  factor  (Devore,  1991:371). 
This  study  investigated  three  separate  factors.  They  were: 
(1)  type  of  display  device,  (2)  subject  experience  level, 
and  (3)  type  of  task.  For  the  purpose  of  defining  the  two- 
level  factorial  experiment,  each  factor  was  evaluated  at  a 
high  (■(-)  and  a  low  (-)  level.  The  display  device  was  either 
an  HMDD  system  (+)  or  a  PMA  system  (-) .  Subject  experience 
level  was  categorized  as  experienced  (+)  or  less-experienced 
(-) .  The  type  of  task  was  categorized  as  procedural  (-f)  or 
inspection-oriented  (-) .  An  example  of  a  procedural  task 
would  be  a  step-by-step  test  procedure  on  a  system  in  which 
the  technician  would  be  required  to  recognize  when  hardware 
responses  deviate  from  normal  operating* ranges.  An  example 
of  an  inspection  oriented  task  would  be  a  visual  inspection 
of  a  system  or  systems.  The  technician  would  be  provided 
with  a  checklist  specifying  the  hardware  items  to  visually 
inspect  and  the  properties  that  would  cause  the  items  to  be 
identified  as  discrepant. 

Sample  size  was  constrained  by  the  availability  of 
technicians  certified  in  the  selected  tasks.  The 
experimental  plan  was  for  three  replications  of  the  eight 
runs  required  to  fully  evaluate  the  three  factors. 

Therefore,  each  technician  would  perform  one  trial  for  a 
total  of  24  runs.  (However,  five  additional  trials  were 
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ultimately  accomplished  as  more  technicians  became  available 
for  testing,  resulting  in  a  total  of  29  runs.) 

Factorial  Fundamentals 

A  few  definitions  are  necessary  to  understand  the 
process  of  analyzing  a  factorial  experiment.  Factors  and 
levels  have  been  discussed  above.  The  two  types  of  results 
that  are  calculated  using  data  obtained  from  a  factorial 
experiment  are  main  effects  and  interaction  effects.  An 
interaction  between  factors  occurs  if  the  effect  of  one 
factor  is  not  the  same  at  all  levels  of  the  other  factors 
(Montgomery,  1976:  197).  A  main  effect  is  the  average 
effect  of  that  factor  over  all  conditions  of  the  other 
variables  (Box,  1978:309).  An  interaction  effect  is  defined 
as  the  difference  in  response  between  the  levels  of  one 
factor  due  to  the  varying  of  the  other  factors  (Montgomery, 
1976:  122). 

Interactions  occur  when  the  effect  of  one  variable  is 
not  completely  independent  of  other  variables.  That 
is,  when  the  effect  of  one  variable  on  the  outcome  of 
the  experiment  is  a  function  of  the  level  of  the  other 
variables.  (Pappas,  1991:49) 

Interaction  effects  play  a  major  role  in  evaluating  the  main 
effects.  "A  significant  interaction  can  tend  to  mask  the 
significance  of  main  effects"  (Montgomery,  1976:  123) .  As  a 
result,  "the  main  effect  of  a  variable  should  be 
individually  interpreted  only  if  there  is  no  evidence  that 
the  variable  interacts  with  other  variables"  (Box,  1978: 

317) . 
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Finally,  the  design  matrix  is  the  medium  that  draws  the 

factors,  their  levels,  the  main  effects,  and  the  interaction 

effects  together.  Montgomery  (1976:188)  writes: 

The  following  table  illustrates  the  treatment 
combinations  required  to  evaluate  fully  the  first  order 
effects  of  a  2^  factorial  design.  Each  factor  A,  B, 
and  C  is  evaluated  at  two  levels,  a  high  and  low  level. 
These  levels  are  represented  by  (-f)  for  high  and  (-) 
for  low  and  can  be  (quantitative  or  qualitative  in 
nature . 

2®  FACTORIAL  DESIGN  MATRIX 


Treatment  Factorial  Effect 


Combination 

I 

A 

B 

AB 

C 

AC 

BC 

ABC 

1 

+ 

- 

- 

+ 

- 

+ 

+ 

- 

2 

+ 

+ 

- 

- 

- 

- 

+ 

+ 

3 

+ 

- 

+ 

- 

- 

+ 

- 

+ 

4 

+ 

+ 

+ 

+ 

- 

- 

- 

- 

5 

+ 

- 

- 

+ 

+ 

- 

- 

+ 

6 

+ 

+ 

- 

- 

+ 

+ 

- 

- 

7 

+ 

- 

+ 

- 

+ 

- 

+ 

- 

8 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

The  heading  for  each  column  represents  the  effect  being 
evaluated.  A  factor  effect  is  represented  by  the  change  in 
the  output  variable  response  when  that  factor  is  varied  from 
the  low  level  (-)  to  the  high  level  (+)  (Box,  1978:309).  An 
effect  is  considered  significant  when  its  absolute  value  is 
significantly  different  from  zero.  The  I  term  is  a  measure 
of  the  mean  response.  The  A,  B,  and  C  terms  represent  the 
main  effects,  which  are  calculated  by  subtracting  the 
average  value  of  all  results  for  which  the  particular 
variable  was  low  (-)  from  those  for  which  the  particular 
variable  was  high  (-*■) .  The  AB,  AC,  BC,  and  ABC  terms  are 
called  the  interaction  effects.  The  calculation  of  the 
interaction  effects  was  the  same  as  for  the  main  effects.  A 
low  level  interaction  effect  was  represented  when  the 
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product  of  the  levels  for  the  associated  factors  was 
negative.  A  high  level  was  represented  when  the  product  of 
the  levels  for  the  associated  factors  was  positive.  For 
example,  both  (+,-)  and  (-,+,+)  would  be  representative  of  a 
low  lev'  1  Interaction.  Conversely,  an  example  of  a  high 
level  Interaction  would  be  represented  by  a  (-,-)  or 
(+»“»-)•  (Box,  1978:310) 

Each  row  In  the  matrix  represents  an  experiment  with 
the  conditions  specified  by  the  (+)  or  (-)  under  the 
appropriate  column  factor.  For  example,  the  first  trial  In 
the  matrix  would  be  based  on  having  all  the  factors  at  their 
low  (-)  level. 

To  determine  If  the  effect  Is  significant,  the  variance 
of  the  effect  must  be  determined.  Generally,  the  variance 
of  the  effect  can  be  expressed  as: 

Var  (effect) -^o^/N  (1) 

where  Is  approximated  by  S^,  the  pooled  estimate  of  the 
run  variance,  and  N  Is  the  total  number  of  runs.  This 
formula  applies  when  the  sample  sizes  (or  degrees  of 
freedom)  for  each  row  (design  point)  are  equal.  (Box, 
1978:320) 

Obtaining  a  pooled  estimate  of  the  variance  requires 

that  runs  be  replicated  at  one  or  more  design  points. 

In  general.  If  g  sets  of  experimental  conditions  are 
genuinely  replicated  and  the  n^  replicate  runs  made  at 
the  1*^^  set  yield  an  estimate  of  having  equals 
ni-1  degrees  of  freedom,  the  pooled  estimate  of  run 
variance  Is 
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(2) 


-2- 

V1+V2+-+V5 

with  V  -  1^1+  1/2+...+  p,  degrees  of  freedom. 

(Box,  1978:319) 

The  pooled  estimate  can  then  be  used  as  an  approximation 
of  the  total  variability,  ,  affecting  runs  made  at  the 
different  experimental  conditions.  The  variance  of  the 
effect  can  be  determined  with  Equation  1  using  the  pooled 
estimate  of  the  run  variance  and  the  total  number  of  trials 
performed.  The  standard  error  of  the  effect  can  be  found  by 
taking  the  square  root  of  the  variance  of  the  effect.  (Box, 
1978:319)  The  level  of  significance  was  set  at 
approximately  .05  using  2a  as  the  critical  value. 

Experimental  Subjects  and  Tasks 

The  178th  TFG,  Springfield,  GANG,  which  flies  the  A-7D 
attack  aircraft,  was  selected  to  support  this  experiment. 

The  organization  selected  had  to:  (1)  be  representative  of 
an  operational  flying  unit,  (2)  have  a  sufficient  number  of 
maintenance  technicians  available  to  support  experimental 
requirements,  (3)  have  technicians  with  a  wide  variety  of 
experience  levels,  and  (4)  routinely  perform  flightline 
maintenance  activities.  GANG  at  Springfield  met  these 
requirements.  An  operational  flying  unit  was  selected  to 
increase  the  generalizability  of  the  experiment.  The 
requirement  for  a  wide  variety  of  experience  was  designed  to 
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maximize  the  contrast  between  the  two  levels  of  the 
experience  factor.  Similarly,  the  requirement  for  routine 
performance  of  a  flightline  maintenance  activity  was 
intended  to  increase  the  applicability  of  the  study  to  other 
maintenance  organizations. 

The  selected  tasks  would  be  a  critical  factor  in 
successfully  answering  the  research  question.  The  task 
selection  process  was  conducted  using  subject  matter  experts 
(SMEs) .  SME  technicians  from  the  178th  TFG  OANG  provided  a 
list  of  14  tasks  for  consideration.  The  tasks  were 
inherently  limited  to  those  conducted  by  the  178th,  the 
participating  maintenance  organization. 

The  selection  of  the  tasks  was  based  on  the  following 
characteristics:  (1)  approximate  completion  time  for  the 
task,  (2)  visual  access  to  a  hand-held  computer  while 
performing  the  task,  (3)  the  degree  of  difficulty  in  gaining 
access  to  the  technical  data  displayed  by  the  PMA,  (4)  the 
degree  of  reliance  on  technical  data  required  to  complete 
the  task,  (5)  the  representativeness  of  the  task  to  common 
tasks  performed  in  a  field  environment,  (6)  the  time 
required  to  have  both  hands  free  to  complete  the  task,  (7) 
the  number  of  times  the  candidate  task  is  performed  on  a 
normal  basis,  and  (8)  the  degree  to  which  the  task  can  be 
performed  by  one  technician. 

Maintenance  Evaluation  Board.  After  receipt  of 
candidate  tasks,  a  maintenance  evaluation  board  composed  of 
experts  from  the  Air  National  Guard,  AL  Human  Factors 
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Engineers,  AFIT  faculty  from  the  School  of  Systems  and 
Logistics  and  the  School  of  Engineering,  and  the  current 
researchers  evaluated  the  14  proposed  tasks  based  on  the 
criteria  described  above. 

Some  of  the  maintenance  activity  types  rejected  by  the 
board  were  short-duration  remove-and-replace  activities, 
troubleshooting  activities,  and  tasks  requiring  a  two-man 
team  for  safety  reasons.  Remove-and-replace  tasks  of 
approximately  30  minutes  duration  were  not  selected  because 
they  could  be  completed  without  reliance  on  technical  data. 
Troubleshooting  tasks  were  not  selected  because  task 
completion  times  were  driven  by  factors  other  than  the  type 
of  display  device.  Tasks  requiring  two  technicians  due  to  a 
two-man  safety  policy  were  eliminated  because  there  was  an 
insufficient  number  of  technicians  to  support  the  desired 
replications.  The  board  determined  that  inspection  and 
procedural  system  checkouts  were  the  best  suited  of  the 
candidate  tasks  to  evaluate  the  effects  of  experimental 
treatments . 

Procedural  Task  Selection.  The  board  selected  the 
operational  checkout  of  the  A-7D's  APQ-126  radar.  For  the 
remainder  of  this  study,  this  task  will  be  referred  to  as 
the  Fault  Isolation  Test.  The  technical  data  for  the  task 
was  extracted  from  T.O.  1A-7D-2-14-3 .  The  task  technical 
data  was  tailored  to  reduce  the  task  completion  time  to 
approximately  30  minutes.  The  data  for  this  test  is  sho%m 
in  Appendix  E.  Activities  outside  the  cockpit  and 
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activities  requiring  a  second  technician  were  either 
eliminated,  as  previously  mentioned,  or  performed  by  the 
current  researchers.  The  tailoring  resulted  in  a  task  that 
could  be  performed  by  a  single  technician  seated  in  the 
cockpit. 

In  the  Fault  Isolation  Test,  the  researchers 
disconnected  a  co-axial  cable  on  the  radar  assembly  under 
the  nose  radome  of  the  aircraft  (see  Figure  1) .  The 
disconnected  cable  was  not  within  the  test  subject's  view  at 
any  time  prior  to  or  during  the  experiment.  The  fault  could 
only  be  detected  by  following  the  technical  data  displayed 
and,  hence,  provided  a  means  of  assessing  how  closely  the 
test  subjects  followed  the  technical  data.  The  number  of 
participants  who  missed  the  inserted  fault  was  recorded  and 
used  as  a  measurement  of  technician  performance. 


Figure  1.  Location  of  Fault  Isolation  Test  Fault 
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Inspection  Task  Selection.  The  board  selected  the 


inspection  of  the  A-7D's  engine  bay  to  represent  the 
inspection  task.  This  task  will  be  referred  to  as  the 
Engine  Bay  Inspection.  The  engine  bay  inspection  is 
performed  on  the  aircraft's  structure  and  subsystems  as  part 
of  the  aircraft  phase  inspection.  During  the  phase 
inspection,  the  Allison  TF*-41  engine  is  removed  to  allow 
access  to  the  compartment.  Tailoring  of  the  inspection  task 
reduced  task  completion  time  to  approximately  30  minutes  and 
limited  personnel  to  one  specialty.  This  checkout  normally 
involves  several  different  specialty  shops,  such  as 
Hydraulic,  Electrical,  Instrumentation,  Environmental, 
Propulsion,  and  Aircraft  Personnel,  General  (APG) .  In  the 
current  research,  the  inspection  was  performed  by  APGs  in 
accordance  with  workcards,  T.O.  1A-7D-6WC-5.  Specifically, 
the  three  cards  selected  were  1-006  (fuel  system),  1-014 
(throttle  cables) ,  and  1-016  (fire  detection  system) .  The 
tailored  data  is  shown  in  Appendix  E.  All  three  subsystem 
inspections  were  performed  inside  the  engine  bay. 

In  the  Engine  Bay  Inspection,  the  inserted  fault  was  a 
fire  detection  sensor  wire  loosened  from  the  retainer  clamps 
(see  Figure  2) .  The  inserted  fault  was  an  obvious 
discrepancy  among  many  already  existing  in  the  test  article 
engine  bay.  The  validity  of  the  numerous  existing  faults 
was  verified  by  a  Quality  Assurance  Inspector  from  the  178th 
Organizational  Maintenance  Squadron.  As  with  the  Fault 
Isolation  Test,  the  inserted  discrepancy  was  input  to  help 
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Figure  2.  Engine  Bay  Inspection  Inserted  Fault 

determine  how  closely  test  subjects  followed  the  technical 
data  displayed. 


Tailoring  the  procedures  provided  an  added  benefit. 
Technicians  with  previous  exposure  to  the  tasks  could  not 
perform  the  tailored  tasks  without  relying  on  technical 
data. 


Experimental  Controls 

Precautions  were  taken  in  order  to  obtain  valid  data. 
The  types  of  controls  placed  on  the  experiment  fell  into  two 
categories:  those  affecting  the  task  completion  times  and 
those  affecting  the  performance  of  the  tasks. 

Completion  Time  Controls.  Task  completion  times  did 
not  depend  on  the  technician  finding  the  inserted  fault. 
Since  the  number  of  test  subjects  were  limited,  this 
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provided  a  reasonable  assurance  that  each  experimental  run 
would  yield  valid  data.  Task  completion  time  was  determined 
by  observing  when  the  technician  completed  all  of  the  steps 
displayed  on  the  last  screen.  To  reduce  possible  error  in 
task  time  measurement,  two  timers  were  used  to  record  the 
task  completion  time.  The  times  were  averaged  for  each  run 
unless  discrepancies  in  performing  the  timing  were  apparent. 

The  PMA  provided  for  this  experiment  was  an  early 
prototype.  As  a  result,  the  screen  retrieval  times  were 
substantially  different  between  the  HNDD  and  PMA.  This  was 
not  a  constraint  of  flat-screen  systems  in  general. 
Subsequent  PMA  models  have  comparable  processing  times  to 
the  HMDD.  Screen  retrieval  times  would  have  been 
essentially  the  same  for  the  PMA  used  in  this  study  if  it 
had  been  constructed  with  a  comparable  Central  Processing 
Unit  (CPU) .  Therefore,  adjustments  to  recorded  task 
completion  times  were  made  by  the  researchers  to  insure  the 
experimental  results  did  not  reflect  any  differences  due  to 
the  separate  computer  CPUs.  These  adjustments  were 
accomplished  by  taking  time  measurements  of  the  HMDD  and  PMA 
response  times  in  retrieving  T.O.  data  to  their  screens. 
Retrieval  rates  were  dependent  on  the  amount  of  text  to  be 
displayed.  Therefore,  each  technician's  selection  of 
screens  during  the  experimental  run  was  recorded.  Of 
particular  importance  were  the  instances  in  which  the 
technicians  backed  up  to  view  the  previous  screen.  The  task 
completion  times  for  each  experimental  run  were  then 
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corrected  to  reflect  only  the  time  displayed  data  was 
available  to  the  technician.  This  was  accomplished  by 
subtracting  out  screen  retrieval  times. 

Task  Controls.  Each  technician  was  required  to  perform 
exactly  the  same  steps.  Therefore,  technicians  who 
identified  the  fault  completed  the  same  number  of  steps  as 
technicians  who  did  not  identify  the  fault.  Placement  of 
the  computers  was  consistent  for  all  runs.  The  PMA  was 
placed  on  a  tool  tray  for  the  Fault  Isolation  Test  (see 
Figure  3)  and  on  the  ground  beneath  the  engine  bay  for  the 
Engine  Bay  Inspection. 

Maintenance  activities  selected  were  tailored  to  insure 
that  the  test  instructions  were  not  the  same  as  the  T.O.s 
from  which  they  were  extracted.  For  instance,  in  the  Fault 
Isolation  Test,  the  frequency  check  was  performed  in  a 


Figure  3.  PMA  Placement  for  Fault  Isolation  Test 
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nonsequential  order  to  increase  the  cognitive  requirements 
of  the  test. 

Presentation  of  the  task  instructions  was  standardized 
for  all  subjects  and  verified  by  three  pilot  studies.  The 
pilot  studies  were  performed  using  subjects  from  the 
intended  test  population. 

To  reduce  the  potential  for  bias;  briefing,  training 
and  debriefing  instructions  (in  the  form  of  prepared 
scripts)  were  issued  before  and  after  the  experimental  runs 
to  all  test  subjects  (see  Appendices  F,  G,  and  H) . 

Test  subjects  were  concentrated  into  four  separate 
groups.  The  less-experienced  and  experienced  technicians 
from  the  GANG  avionics  shop  comprised  two  groups.  The  less- 
experienced  and  experienced  technicians  from  the  GANG  APG 
shop  comprised  the  other  two  groups.  The  less-experienced 
technicians  were  the  "traditional  guardsmen"  who  performed 
GANG  duties  two  days  a  month  and  during  a  two-week  period 
once  a  year.  Experienced  technicians  were  full-time  GANG 
employees.  Care  was  taken  in  evaluating  the  amount  of 
expertise  each  guardsmen  possessed.  If,  for  example,  a 
traditional  guardsmen  had  a  background  similar  to  a  full¬ 
time  technician's  experience  on  the  system  being  tested,  the 
technician  was  reclassified  as  experienced.  All  the 
technicians  were  randomly  assigned  to  the  type  of  display 
used  in  performing  the  trial. 

The  current  researchers  performed  all  of  the  data 
collection  activities  to  reduce  data  recording  error  and. 
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more  importantly,  to  insure  consistent  qualitative 
observations.  Finally,  the  technical  data  displayed  and 
software  used  to  present  the  data  on  both  the  HMDD  and  PMA 
were  identical. 

Calculating  the  Effects 

To  better  understand  how  the  quantitative  findings  were 
obtained,  the  following  section  will  outline  the  process 
used  in  calculating  the  effects  for  the  task  completion 
times  of  the  total  sample. 

The  first  step  was  obtaining  and  recording  the  raw 
completion  times  for  the  29  trials  performed  (see  Appendix 
I) .  Times  initially  recorded  as  minutes  and  seconds  were 
converted  to  minutes  and  tenths  of  minutes  to  simplify 
calculations.  The  difference  in  response  rates  was  then 
subtracted  out  as  previously  explained.  This  was  done 
individually  for  each  trial  since  some  test  subjects 
performed  a  different  number  of  screen  retrievals.  The 
recorded  sequence  of  data  retrieval  (including  backing  up  to 
review  data)  was  compiled  to  determine  the  time  differential 
for  each  trial.  Corrected  data  can  be  found  in  Appendix  J. 
The  next  step  was  to  gather  descriptive  statistics  of  the 
sample  data.  These  statistics  are  shown  in  Table  1. 

Each  trial  was  performed  at  a  prescribed  level  of  each 
factor  being  investigated.  The  level  of  each  factor  was 
represented  in  the  standard  factorial  matrix  by  a  -fi  or  -1. 
If  the  trial  involved  a  high  level  of  the  factor,  the  matrix 
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Table  1 

Descriptive  Statistics  of  Task  Completion  Times 


Exp  1 

HMDD  1 
Insp  -1 

Exp  1 

PMA  -1 
Insp  -1 

Exp  1 

HMDD  1 

Test  1 

Exp  1  1 

PMA  -1  1 

Test  1  1 

MEAN 

22.55 

25.20 

13.28 

14.33  1 

VARIANCE 

43.16 

15.51 

1.08 

9.77  1 

LesExp  *1 
HMDD  1 
Insp  -1 

WsSmBM 

LesExp  -1 
HMDD  1 
Test  1 

LesExp-1  1 
PMA  -1  1 

Test  1  1 

1  MEAN 

23.78 

19.75 

16.93 

23.99  1 

1  VARIANCE 

45.16 

8.50 

18.15  1 

designator  was  a  +1.  If  the  trial  involved  a  low  level  of 
the  factor,  the  matrix  designator  was  a  -1.  Table  2 
provides  the  conversion  from  category  to  matrix 
representation . 

Table  3  is  a  representation  of  the  factorial  matrix 
used  in  this  study  to  calculate  the  effects  of  each  factor 


Table  2 

Category  to  Matrix  Conversion 
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Table  3 

Factorial  Matrix  for  Task  Completion  Times 


MEAN 

EXP 

DISP 

TASK 

EXP 

OISP 

EXP 

TASK 

DISP 

TASK 

EXP 

DISP 

TASK 

AVG 

OBS 

1 

-1 

-1 

-1 

1 

1 

1 

-1 

19.75 

1 

1 

-1 

-1 

-1 

-1 

1 

1 

25.20 

1 

-1 

1 

-1 

-1 

1 

-1 

1 

23.78 

1 

1 

1 

-1 

1 

-1 

-1 

-1 

22.55 

1 

-1 

-1 

1 

1 

-1 

-1 

1 

23.99 

1 

1 

-1 

1 

-1 

1 

-1 

-1 

14.33 

1 

-1 

1 

1 

-1 

-1 

1 

-1 

16.93  1 

1 

1 

1 

1 

1 

1 

1 

13.28 

8 

4 

4 

4 

4 

4 

4 

4 

-19.97 

-2.27 

-1.68 

-5.69 

-0.17 

-4.39 

-2.37 

and  interaction  term  for  the  task  completion  times  of  the 
total  sample.  At  the  top  of  the  matrix  are  the  various  main 
and  interaction  effects  being  investigated.  The  first  term, 
MEAN,  is  an  average  value  and  is  useful  in  scaling  the 
importance  of  the  effects  values.  EXP  refers  to  the  main 
effect,  level  of  experience,  and  its  influence  on  the  total 
task  completion  time.  DIS  refers  to  the  display  type  main 
effect.  TSK  refers  to  the  main  effect  for  the  type  of  task 
performed.  In  addition,  three  two-factor  interactions  and 
one  three-factor  interaction  term  are  displayed  at  the  top 
of  the  2^  factorial  design  matrix.  The  sample  means  were 
placed  adjacent  to  the  row  in  the  factorial  matrix  that 
represents  the  conditions  under  which  the  output  data  was 
collected.  For  example,  in  the  first  row  of  the  table. 
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19.75  was  the  mean  completion  time  for  less-experienced 
technicians  using  the  PMA  for  the  Engine  Bay  Inspection. 

Table  3  provides  an  arithmetic  guide  in  calculating 
each  of  the  effects.  Calculating  the  effect  of  a  factor  is 
a  simple  matter  of  (1)  assigning  the  appropriate  sign  (+,-) 
of  the  effect  to  the  observed  value  for  that  row,  (2) 
summing  the  modified  observation  column,  and  (3)  dividing 
this  sum  by  the  divisor  displayed  in  the  next-to-last  row  of 
the  column. 

In  order  to  determine  the  significance  of  an  estimated 
effect,  an  estimate  of  its  variance  needs  to  be  determined 
as  previously  described.  First,  a  pooled  estimate  of  the 
run  variance  is  obtained  using  Equation  2  as  described  in 
the  factorial  fundamentals  section  of  this  paper.  Then  the 
variance  of  the  effect  is  calculated  using  the  pooled 
estimate  of  the  run  variance  and  the  number  of  trials. 

This  can  be  performed  in  one  of  two  ways  depending  on 
the  type  of  sample  sets  being  used.  If  the  sample  sets  are 
balanced,  in  other  words,  the  number  of  observations  at  each 
design  point  are  equal,  Ec[uation  1  can  be  used.  If  the 
sample  sizes  are  not  balanced,  another  expression  of  the 
same  concept  must  be  used.  Since  the  estimate  is  a  linear 
combination  of  the  observed  values  (which  are  random 
samples) ,  the  variance  of  the  effect  is  the  variance  of  the 
linear  combination.  As  a  result,  the  variance  of  effect 
expression  is  dependent  upon  the  number  of  design  points  and 
replications  for  each  sample  set.  In  the  case  of  the  task 


41 


completion  times  for  the  total  sample  used  in  this  r^ssearch, 
the  variance  of  the  effect  would  be  expressed  as; 


Var  (effect) 


_  3  (0^/3)  •t^5(oV4) 
16 


(3) 


This  expression  is  based  on  a  total  sample  consisting  of 
three  design  points  containing  three  replications  and  five 
design  points  containing  four  replications.  The  pooled 
estimate  of  the  run  variance  (Equation  2)  was  still  used  for 
an  approximation  of  a^.  Table  4  contains  the  variance 
values  calculated  for  the  task  completion  times  of  the  total 
sample. 


Table  4 

Estimation  of  Standard  Error 


Pooled  Estimate 

21.57 

Variance  of  Effect 

2.98 

Standard  Error 

1.73 

2*  Sigma 

3.46 

Effect  significance  was  determined  by  evaluating  the 
effect's  value  compared  to  two  times  the  standard  error  (the 
square  root  of  the  variance  of  the  effect) .  This  level  of 
significance  is  approximately  equal  to  a  95%  level  of 
confidence. 
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Suminarv 


This  chapter  introduced  the  method  used  to  evaluate  the 
data  collected  and  the  rationale  for  selecting  the 
experimental  design,  the  tasks  performed,  and  experimental 
controls.  The  main  factors  used  in  the  design  were  display 
type,  technician  experience  level,  and  task  type.  In  the 
next  chapter,  the  experimental  findings  are  presented. 
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IV.  Findings 


Overview 

This  study  used  two  quantitative  measures  (task 
completion  times  and  number  of  faults  found)  and  two 
qualitative  measures  (survey  responses  and  interview 
responses)  to  evaluate  technician  performance.  The 
experiment  described  in  Chapter  3  was  conducted  to  obtain 
task  completion  times  to  quantify  the  efficiency  of 
technician  performance.  The  number  of  faults  found  was 
collected  to  quantify  the  effectiveness  of  technician 
performance.  Post  trial  survey  and  interview  responses  were 
used  to  evaluate  technician  perceptions  of  the  display 
devices. 

This  chapter  was  structured  to  address  the  measures 
used  in  the  study:  completion  times,  faults  found,  survey 
responses,  and  interview  responses.  These  measures  are 
preceded  by  a  description  of  the  actual  testing  environment 
to  provide  a  better  understanding  of  findings.  The  analysis 
of  the  completion  times  was  broken  into  three  categories; 

(1)  task  completion  time  for  the  Fault  Isolation  Test,  (2) 
task  completion  time  for  the  Engine  Bay  Inspection,  and  (3) 
task  completion  time  for  the  total  sample.  Analysis  of  the 
nximber  of  faults  found  was  broken  into  two  categories; 

(1)  number  of  faults  found  for  Engine  Bay  Inspection  and  (2) 
missed  detection  of  inserted  faults  for  both  tasks. 
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How  the  Experiment  was  Performed 


Thirty-one  experimental  runs  were  attempted  over  a 
period  of  three  weeks.  Twenty-nine  experiments  were 
successfully  conducted.  One  experiment  was  aborted  due  to  a 
technician's  inability  to  read  the  HMDD  screen.  The 
technician  was  farsighted,  normally  wore  reading  glasses, 
and  did  not  have  them  available  at  the  time  of  the  test. 

The  other  experiment  was  aborted  because  a 

less-experienced  technician  could  not  correctly  identify  the 
cockpit  switches  needed  to  perform  the  task.  Both 
technicians  did  complete  the  questionnaire  and  interview 
process . 

The  31  technicians  came  from  two  different  shops.  The 
avionics  shop  personnel  performed  a  portion  of  the  Fault 
Isolation  Test  of  the  A-7D  APQ-126  radar  system.  The 
personnel  from  the  APG  shop  performed  a  portion  of  an  Engine 
Bay  Inspection  normally  conducted  on  the  A-7D  airframe 
during  a  phase  inspection.  Each  maintenance  technician 
performed  his  associated  maintenance  task  once. 

Because  of  the  limited  availability  of  traditional 
guardsmen,  the  testing  sequence  was  organized  to  test 
homogenous  groups  of  subjects.  For  instance,  all  the  less- 
experienced  avionics  technicians  performed  the  procedural 
task  during  a  two-day  weekend.  Testing  of  all  less- 
experienced  APG  technicians  performing  the  inspection  task 
was  conducted  during  the  following  two-day  weekend.  Testing 
of  experienced  technicians  was  conducted  over  a  six-day 
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period,  with  avionics  and  APG  technicians  tested  on  separate 
days.  This  testing  sequence  was  necessary  to  minimize  the 
impact  on  OANG  workload. 

The  type  of  display  used  during  each  trial  was  randomly 
selected  at  the  beginning  of  the  day.  Some  substitutions 
were  made  during  the  testing  day  when  either  both  sets  of 
HMDD  batteries  required  recharging  or  the  PMA  required  a 
cooling-down  period.  The  internal  components  of  the 
prototype  PMA  heated  up,  causing  the  display  screen  to  lock. 
To  free  up  the  screen,  the  PMA  had  to  be  turned  off  until 
the  components  had  cooled  down. 

Some  visual  problems  were  found  in  using  the  HMDD  with 
both  less-experienced  and  experienced  technicians. 
Specifically,  the  HMDD  test  hardware  was  not  easily 
adaptable  for  the  technicians  with  farsighted  vision  or  for 
the  ones  who  wore  bifocal  corrective  lenses.  Even  though 
the  visual  problems  did  not  adversely  effect  the  performance 
of  any  participant,  the  limited  adaptability  of  the  HMDD 
used  in  this  study  adversely  influenced  the  technician's 
perceptions. 

Task  Completion  Times  for  the  Fault  Isolation  Test 

Figure  4  displays  the  factor  effects  on  the  task 
completion  times  for  the  Fault  Isolation  Test.  The  effects 
were  determined  using  a  2^  factorial  analysis  on  13  of  the 
29  total  trials.  The  factorial  analysis  can  be  found  in 
Appendix  K.  Two  times  the  standard  error,  indicated  by  the 
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2  *  sigma  in  the  figure,  was  used  as  a  benchmark  to  gauge 
the  factor  effects'  significance.  The  average  length  of 
time  required  to  complete  the  Fault  Isolation  Test  was  17.13 
minutes. 

Main  Effects.  Both  main  effects  (experience  and 
display  type)  were  considered  statistically  significant. 

The  experimental  results  suggest  the  experience  level  had 
more  of  an  influence  in  reducing  the  completion  times  than 
display  type  used  (-6.66  minutes  versus  -4.06  minutes).  A 
negative  effect  is  an  indication  that  the  high-level  factor 
had  a  shorter  time  than  the  low-level  factor.  In  this  case, 
the  experienced  technicians  were  able  to  complete  the  task 
faster  than  the  less-experienced  technician  on  average. 
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Technicians  using  the  HMDD  finished  the  task  quicker  on 
average  than  their  counterparts  using  the  PMA. 

Experience /Display  Interaction  Effect.  The 
experience/display  interaction  effect  was  slightly  less  than 
the  2a  threshold.  The  researchers  deemed  that  the 

I 

interaction  effect  was  significant  enough  to  warrant 
discussion.  Note  the  points  representing  completion  times 
in  Figure  5.  The  points  for  the  less>experienced 
technicians  are  spaced  farther  apart  than  the  points  for  the 
experienced  technicians.  The  less-experienced,  HMDD- 
equipped  technicians  performed  the  task  7.06  (23.99  -  16.93) 
minutes  faster  than  their  PMA-equipped  counterparts  on 
average.  The  time  difference  resulting  from  experienced 
technicians  performing  the  Fault  Isolation  Test  favored  the 


—  HMDD  — PMA 


Figure  5.  Experience/Display  Interaction  for  Fault 
Isolation  Test 
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use  of  the  HMDD  over  the  PMA  by  1.05  (14.33  -  13.28)  minutes 
on  average.  The  HMDD  tended  to  influence  the  less- 
experienced  technicians'  task  completion  time  on  the  Fault 
Isolation  Test  more  than  that  of  the  experienced 
technicians.  This  observation  was  confirmed  by  positive 
sign  of  the  experience/display  interaction  effect  shown  in 
Figure  4. 

Task  Completion  Times  for  the  Engine  Bay  Inspection 

Figure  6  displays  the  factor  effects  on  the  task 
completion  times  for  the  Engine  Bay  Inspection.  The 
associated  2^  factorial  analysis  was  based  on  16  trials  and 
is  located  in  Appendix  L.  On  average,  the  technicians 
performed  the  inspection  in  22.82  minutes.  None  of  the 
factors  were  considered  significant. 


Figure  6.  Task  Completion  Time  for  Engine  Bay  Inspection 
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Task  Completion  Tiroes  for  the  Total  Sample 


Figure  7  displays  the  results  of  the  factorial  analysis 
performed  on  the  total  sample  of  completion  times.  Unlike 
the  previous  factorial  analysis,  the  total  sample  factorial 
used  a  2^  design.  The  analysis  can  be  found  in  Chapter  3. 


Figure  7.  Task  Completion  Time  for  Total  Sample 
Three  main  effects  and  four  interaction  effects  were 
evaluated  based  on  all  29  trials  conducted  during  the 
experiment.  The  task  main  effect  and  the  experience/task 
interaction  were  the  only  effects  that  met  the  significance 
benchmark . 

Task  Main  Effect.  The  task  main  effect  confirms  that 
the  Engine  Bay  Inspection  task  took  less  time  to  complete 
than  the  Fault  Isolation  Test. 
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Experience /Task  Interaction.  The  experience/ task 
interaction  effect  provides  an  indication  of  how  the  task 
performed  can  influence  the  effect  experience  has  on  a 
technician's  performance.  Shown  in  Figure  8  is  a  graphical 
representation  of  this  influence  on  technician  performance. 

Less-experienced  technicians  performing  the  Engine  Bay 
Inspection  completed  the  task  2.11  (23.88  -  21.77)  minutes 
faster  than  the  experienced  technicians  on  average.  The 
trend  is  reversed  for  the  Fault  Isolation  test.  Experienced 


“^Engine  Bay  Checkout  H— Radar  Checkout 

Figure  8.  Experience/Task  Interaction  for  Task  Completion 
Time 

technicians  performed  the  task  6.65  (20.46  -  13.81)  minutes 
faster  on  average  than  their  less-experienced  counterparts. 
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The  second  measure  used  to  evaluate  technician 
performance  was  the  number  of  faults  found.  The  Engine  Bay 
Inspection  was  unique  because  there  were  numerous  faults  in 
each  of  the  subsystems  inspected  in  the  engine  bay  (in 
addition  to  the  inserted  fault) .  The  associated  2^ 
factorial  analysis  located  in  Appendix  M  was  based  on  16 
trials.  The  results,  shown  in  Figure  9,  indicate  that  on  the 
average,  technicians  found  eight  faults.  The  display  type 
main  effect  appeared  to  significantly  influence  technician 
performance.  The  experience/display  type  interaction  effect 
was  slightly  less  than  the  2a  benchmark  but  warrants 
discussion. 


Figure  9.  Number  of  Faults  Found  for  Engine  Bay  Inspection 
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.  The  display  type  effect  favors 


the  use  of  the  HMDD  over  the  PMA  for  finding  more  faults  in 
an  inspection  task.  Technicians  using  the  HMDD  tended  to 
find  five  more  faults  on  average  than  their  PMA  equipped 
counterparts . 

Experience/Displav  Interaction  Effect.  In  the  previous 
discussion  about  the  experience/display  interaction  for  the 
Fault  Isolation  Test  completion  times,  the  findings 
suggested  that  less-experienced  technicians  benefitted  the 
most  from  using  the  HMDD.  However,  with  respect  to  the 
number  of  faults  found,  the  findings  suggest  that 
experienced  technicians  benefitted  more  from  the  use  of  the 
HMDD  than  did  the  less-experienced.  The  graphical 
representation  of  this  effect  is  shown  in  Figure  10.  The 


»  of  Faults 


-^HMDD  — PMA 


Figure  10.  Experience/Display  Interaction  Effect  for 
Engine  Bay  Inspection  Faults  Found 


53 


difference  in  the  number  of  faults  found  by  the  experienced 
technicians  was  8.5  (13.5  -  5).  The  difference  in  the 
number  of  faults  found  by  the  less-experienced  technicians 
was  two  (8.5  -  6.5).  Note  that  the  number  of  possible 
faults  remained  constant  throughout  the  experiment.  Each 
technician  had  the  same  opportunity  to  find  faults  as  the 
next . 


Detection  of  Inserted  Faults  for  Both  Tasks 

Each  of  the  tasks  had  one  inserted  fault.  In  Appendix 
J,  the  Missed  Detection  of  Inserted  Faults  Table  shows  the 
total  number  of  technicians  who  missed  the  inserted  faults. 
Shown  in  Table  5  below  is  the  breakout  of  missed  inserted 


Table  5 

Breakout  of  Missed  Inserted  Faults 


P  Group 

1  All  Subjects 

8/29 

28%  1 

1  HMDD 

3/14 

21%  1 

1 

5/15 

33%  1 

1  Experienced 

3/15 

20% 

Le s s - Exper i enced 

5/14 

36% 

Fault  Isolation  Test 

4/13 

31% 

Engine  Bay  Inspection 

4/16 

25% 

faults  by  factors.  When  considering  display  devices,  12% 
more  of  the  technicians  using  the  HMDD  found  the  inserted 
fault  than  the  technicians  using  the  PMA.  When  considering 
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experience  level,  16%  more  of  the  experienced  technicians 
found  the  inserted  fault  than  the  less-experienced 
technicians.  When  considering  task  type,  the  Engine  Bay 
Inspection  fault  was  found  6%  more  often  than  the  fault  in 
Fault  Isolation  Test. 


Faults  and  Completion  Times  Interaction 

Factors  that  improve  technician  effectiveness  are  often 
expected  to  improve  efficiency  as  well.  This  relation  was 
not  the  case,  however,  for  the  Engine  Bay  Inspection  task. 
The  relation  of  the  measure  used  for  effectiveness  (total 
number  of  faults  found  and  number  of  technicians  that  missed 
the  inserted  fault)  to  the  measure  used  for  efficiency 
(completion  time  for  the  technician  to  complete  the  task 
once  regardless  if  the  fault  was  found)  was  in  most  cases 
dichotomous.  As  previously  discussed,  the  technician  using 
the  HMDD  displayed  more  effective  performance  than  the  PMA 
equipped  technician  by  finding  more  faults  in  the  engine 
bay.  With  respect  to  experience  level,  the  experienced 
technician  performed  more  effectively  (found  more  faults) 
than  the  less-experienced  technician.  However,  the 
efficiency  of  these  groups  was  reversed.  Completion  times 
for  the  Engine  Bay  Inspection  suggested  that  HMDD  equipped 
technicians  were  less  efficient  (took  longer  to  perform  the 
inspection)  than  the  PMA  equipped  technicians.  The 
experienced  technicians  were  likewise  less  efficient  than 
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the  less-experienced  technicians  in  performing  the 
inspection. 

Completion  time  differences  that  were  calculated  using 
data  presented  in  Table  1  bear  this  out.  Technicians  using 
the  HMDD,  on  the  average,  took  more  than  1.38  minutes 
[ (22, 55+23. 78)-(25. 2+19. 75) ]  longer  to  complete  the 
inspection  task  than  PMA-equipped  technicians.  Experienced 
technicians  took  4.22  minutes  [ (22. 55+25. 20} -(23. 78+19. 75) ] 
longer  on  the  average  than  the  less-experienced  technicians 
in  completing  the  Engine  Bay  Inspection  task. 

Survey  and  Interview  Findings 

The  discussion  in  this  section  is  divided  into  three 
areas:  display  device  preferences,  HMDD  visual  aspects,  and 
screen  fading.  A  complete  listing  of  the  questionnaire 
responses  are  contained  in  Appendix  N.  Appendix  O  and 
Appendix  P  contain  the  interview  responses  from  the 
technicians  who  used  the  PMA  and  HMDD  respectively. 

Display  Device  Preferences.  As  discussed  in  Chapter  3, 
each  technician  performed  one  task  using  either  the  HMDD  or 
the  PMA.  As  a  result,  no  technician  could  directly  compare 
the  display  devices.  Rather,  the  information  displayed  in 
Figure  11  is  a  compilation  of  each  technician's  subjective 
assessment  of  the  device  they  used  to  perform  the  trial. 

The  values  shown  are  the  mean  values  of  the  questionnaire 
responses  obtained  from  the  total  sample  of  technicians  who 
used  either  the  HMDD  or  the  PMA.  The  HMDD  was  evaluated  by 
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Text  Readability 
Screen  Glare 
Screen  Brightness 
Screen  Si2e 
Response  Time 
Text  Spacing 


i3.91 


2  3  4 

Questionnaire  mean  values 

■■hMDD  Y//A  PMA 


Figure  ll.  Technicians  Questionnaire  Responses 


16  te<^hnicians,  and  the  PMA  was  evaluated  by  15.  Means  were 
used  here  to  condense  technician  responses  to  provide  the 
reader  with  an  overall  view  of  test  subject  perceptions. 
Appendix  Q  contains  histograms  portraying  the  full  range  of 
responses  on  each  feature  investigated.  The  rating  values 
were  based  on  the  questionnaire's  ordinal  scale  assignments. 
A  5  represented  outstanding,  4  highly  satisfactory,  3 
satisfactory,  2  marginal,  and  1  unsatisfactory. 

Both  display  devices  were  generally  rated  between 
satisfactory  and  highly  satisfactory.  PMA-equipped 
technicians  tended  to  express  more  satisfaction  with  the 
display  device  than  HMDD-equipped  technicians.  Screen  Size 
was  one  area  of  interest  in  which  technicians  using  the  HMDD 
were  noticeably  less  satisfied  than  the  PMA~equipped 
technicians.  The  lack  of  satisfaction  with  screen  size  was 
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attributed  to  the  requirement  to  adjust  the  CRT  location  of 
the  HMOD  in  order  to  read  the  entire  screen.  The  only  area 
rated  marginal  was  the  response  time  of  the  PMA.  The 
researchers  knew  in  advance  that  the  PMA.  had  a  slower 
computer  response  time  and  adjusted  the  task  completion 
times  to  remove  any  biases  caused  by  differing  computer 
processing  times.  The  computer  response  time  problem  was 
inherent  to  the  PMA  used  in  this  experiment  only.  The  most 
recent  PMAs  have  central  processing  units  which  provide 
screen  retrieval  times  comparable  to  the  HMDD. 

Technicians  were  asked  if  there  was  anything  they 
disliked  about  using  the  display  devices.  The  majority  of 
the  responses  addressed  characteristics  specific  to  the 
hardware  used  during  the  trials.  Some  responses  were:  The 
vest  was  too  hot,  the  headset  too  bulky,  or  the  display 
devices  were  too  heavy.  The  vast  majority  of  the 
technicians  felt  that  both  displays  improved  their 
capability  to  perform  maintenance  activities.  However, 
there  was  one  exception:  a  technician  using  the  PMA  said  he 
didn't  see  any  advantage  to  using  a  computer  to  perform 
inspections.  Many  of  the  technicians'  comments  are 
addressed  in  the  Recommendations  section  of  Chapter  5. 

visual  Aspects  of  the  HMDD.  Past  research  suggested 
that  performing  highly  mobile  activities  while  wearing  a 
HMDD  system  might  result  in  visual  difficulties.  Problems 
with  retinal  rivalry,  lack  of  depth  perception,  and  spatial 
disorientation  were  expressed  as  concerns.  In  addressing 
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these  concerns,  this  section  will  provide  a  description  of 
the  type  of  movements  performed  by  technicians  in  this 
study.  Next,  technician  evaluations  with  regard  to  visual 
problems  encountered  are  presented.  Finally,  the 
researchers'  assessment  of  the  visual  problems  is  offered. 

Unlike  Masquelier's  study,  the  technicians  in  this 
study  were  required  to  be  highly  mobile.  The  Engine  Bay 
Inspection  required  more  complex  ambulatory  movements  than 
the  Fault  Isolation  Test.  All  experimental  trials  began  in 
an  office  located  on  the  second  floor  of  the  aircraft 
maintenance  hangar.  There,  technicians  were  aided  in 
putting  on  the  HMDD  equipment  and  the  unit  was  turned  on. 

The  technical  data  for  the  appropriate  task  was  then 
selected.  For  the  trial,  technicians  performing  the  Fault 
Isolation  Test  were  required  to  walk  down  a  flight  of  stairs 
and  across  the  hangar  floor  to  the  aircraft  used.  During 
this  time,  the  HMDD  was  operating  and  displaying  technical 
data.  The  technician  was  then  required  to  climb  a 
maintenance  stand  equipped  with  vertical  stairs  and  climb 
into  the  cockpit  using  the  aircraft's  built-in  steps  (see 
Figure  12) .  The  trial  began  when  the  technician  was  seated 
in  the  cockpit.  Several  times  during  the  trial,  the 
technician  was  required  to  stand  up  in  the  cockpit  and  look 
over  the  canopy  to  view  the  movement  of  the  radar  antenna. 
Throughout  the  experiment,  none  of  the  technicians  exhibited 
or  expressed  difficulty  in  performing  these  actions. 
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Figure  12 .  Fault  Isolation  Test  Environment 


Technicians  performing  the  Engine  Bay  Inspection  were 
required  to  go  through  the  same  initial  sequence  described 
above.  However,  the  APG  technicians  were  required  to  exit 
the  hangar  and  walk  approximately  50  yards  across  the 
aircraft  parking  ramp  to  the  aircraft  used  for  the  trial. 

On  most  occasions,  the  ramp  was  brightly  lit  by  sunlight. 

The  trial  began  once  the  technician  had  arrived  at  the 
aircraft's  aft  end  and  had  obtained  a  flashlight  and  mirror 
required  for  the  inspection  (see  Figure  13) .  During  the 
trial,  the  technician  was  required  to  enter  and  exit  the 
engine  bay  numerous  times.  Each  time,  the  technician  was 
required  to  negotiate  an  approximate  three-foot  step  up  into 
the  engine  bay  while  avoiding  the  tail  hook  positioned 
directly  aft  of  the  aircraft.  While  inside  the  engine  bay, 
the  technician  was  required  to  stoop,  twist,  squat,  lean. 
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Figure  13.  Technician  Performing  the  Engine  Bay  Inspection 

and  stand.  Throughout  the  inspection,  the  technician  was 
required  to  balance  himself  on  aircraft  structural  members 
to  avoid  damaging  aircraft  components  underfoot  (See  Figure 
14) .  None  of  the  technicians  stated  that  they  had 
difficulty  in  performing  these  movements. 

In  the  post-trial  survey,  technicians  assessed  six 
visual  symptoms  that  could  have  resulted  from  using  the 
HMDD.  The  symptoms  were  problems  associated  with  (1) 
switching  attention  from  the  display  to  the  surrounding 
environment  and  back,  (2)  eye  strain,  (3)  blurring,  (4) 
focusing,  (5)  afterimages,  and  (6)  headaches.  Figure  15 
shows  the  technicians'  responses  to  questions  asked  about 
the  visual  aspects  of  the  HMDD. 

On  the  average,  all  of  the  symptoms  except  eye  strain 
were  judged  satisfactory.  Some  test  subjects  did  experience 
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Figure  14.  Engine  Bay  Floor  Configuration 

marginal  results,  however.  Appendix  N  has  the  full  range  of 


responses  provided  by  the  technicians.  A  few  technicians 
(six  out  of  16)  speculated  in  the  interviews  that  if  the 
test  had  been  longer,  they  would  have  experienced  some  eye 
strain.  It  would  seem  likely  that  this  speculation  was  the 
result  of  some  technicians  experiencing  the  onset  of  eye 
fatigue  as  the  result  of  viewing  the  HMDD.  Although  the 
survey  responses  regarding  headaches  appeared  to  be 
satisfactory,  three  less-experienced  technicians  using  the 
HMDD  reported  getting  headaches  several  hours  after 
participating  in  the  experiment.  It  is  worth  noting  that 
the  three  technicians  who  experienced  the  headaches  were  the 
first  technicians  of  the  squadron  to  be  tested  using  the 
HMDD.  Researchers  observed  that  technicians  who  were 
squinting  at  the  beginning  of  the  trial  to  view  the  HMDD 
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screen  were  later  able  to  view  the  screen  with  both  eyes 
open.  This  behavior  was  confirmed  in  the  post-trial 
interviews.  Technicians  stated  that  they  found  it  easier  to 
use  the  HMDD  as  they  became  more  accustomed  to  it. 

Survey  responses,  interview  responses,  and  the 
researchers'  observations  suggest  that  technicians  wearing 
the  HMDD  were  able  to  operate  physically  complex  actions 


Questionnaire  mean  values 
HMDD 

Figure  15.  Visual  Aspects  of  the  HMDD 

involving  balance,  eye-to-hand  coordination,  bending, 
twisting,  and  climbing  under  dim  and  bright  lighting 
conditions  without  suffering  adverse  visual  side  effects 
(with  the  exception  of  the  few  technicians  who  experienced 
slight  eye  fatigue) . 

Screen  Fading.  Fading  of  electronic  media  display 
devices  under  bright  sunlight  conditions  has  been  expressed 
as  a  concern  in  operating  the  HMDD  in  a  flightline 
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environment  (see  Chapter  2) .  Several  areas  of  the  base  were 
evaluated  to  determine  a  location  for  comparing  the  two 
devices  where  they  would  be  most  susceptible  to  screen 
fading.  Light  meter  readings  and  the  corresponding  Lux 
values  obtained  in  various  locations  were  recorded  and  are 
shown  in  Table  6. 


Table  6 

Various  Location  Lux  Value  Readings 


1  Location 

Meter  Reading 

Lux  Value 

QA  Office 

13 

700 

Aircraft  Cockpit 

12.5 

525 

1  Engine  Bay 

10.5 

132 

1  Flight line 

19 

44000 

Based  on  these  results,  the  HMDD  and  the  PMA  were  observed 
under  the  same  light  conditions  at  the  location  where  the 
light  meter  readings  were  the  greatest:  on  the  flightline 
clear  of  the  surrounding  buildings.  The  procedures  used  for 
testing  was  as  follows. 

The  PMA  was  placed  flat  on  the  flightline.  The  PMA 
display  was  assessed  based  on  the  researcher's  ability  to 
stand  three  feet  away  and  read  the  text  displayed  on  the 
screen.  The  PMA  was  evaluated  at  different  elevation  angles 
through  360  degrees  around  the  PMA.  The  researcher  was  able 
to  read  the  text  displayed  on  the  PMA  screen  at  all  the 
viewing  angles  tested. 
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The  HMDD  system  was  evaluated  at  the  same  location. 

The  HMDD  screen  was  assessed  by  having  a  person  wear  the 
system  and  slowly  turn  while  observing  the  text  displayed. 
When  the  wearer  turned  to  face  the  sunlight  directly,  the 
text  was  unreadable.  The  screen  gradually  became  readable 
again  as  the  wearer  turned  his  head  away  from  the  direct 
sunlight.  It  appeared  that  the  sunlight  was  reflecting  off 
the  wearer's  face  and  washing  out  the  text  on  the  screen. 
When  the  same  test  was  conducted  with  the  wearer  having 
untinted  glasses,  the  fading  was  more  pronounced. 

In  an  attempt  to  reduce  the  problem,  a  tinted  polarized 
lens  from  a  35MM  camera  was  placed  in  front  of  the  HMDD 
screen,  and  the  test  was  repeated.  The  fading  was 
significantly  reduced.  With  the  filter  in  place,  the  wearer 
could  read  the  text  while  facing  directly  toward  the  sun. 

It  is  ynclear  whether  the  tinting  or  the  polarization  of  the 
lens  had  the  most  effect  in  reducing  the  screen  fading. 

gyPQPgjs 

The  purpose  of  this  chapter  was  to  inform  the  reader  of 
the  study  findings.  First,  the  actual  conditions 
encountered  during  testing  were  defined  to  provide  the 
reader  with  an  understanding  of  the  specific  conduct  of  the 
study.  A  presentation  of  the  detailed  results  followed  for 
each  of  the  primary  measurements:  completion  times,  detected 
faults,  and  user  perceptions.  The  next  chapter  will  provide 
the  researchers'  overall  discussion  of  the  findings. 
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V.  Discussion  of  Findings 


OYsrview 

The  purpose  of  this  study  was  to  determine  if  there 
were  any  additional  benefits  in  using  a  monocular 
head-mounted  display  device  rather  than  a  hand-held  flat- 
screen  display  device  to  present  maintenance  technical 
orders  to  technicians  performing  flightline  tasks.  By 
drawing  on  the  findings  of  the  previous  chapter,  this 
chapter  will  discuss  the  influence  of  the  following  factors 
on  technician  performance:  (1)  display  type,  (2)  experience 
level,  (3)  type  of  task,  (4)  display  type  and  task  type 
interaction,  (5)  experience  level  and  display  type 
interaction,  (6)  experience  level,  display  type,  and  task 
type  interaction,  and  (7)  participant  perceptions  of  the 
HMDD  and  PMA.  At  the  completion  of  these  specific  factor 
evaluations,  a  general  assessment  of  the  research  question 
will  be  provided. 

DigPlay-TYP? 

The  display  type  used  to  provide  technical  data  made  a 
measurable  difference  in  technician  performance  during  the 
experiment.  For  the  Fault  Isolation  Test,  technicians  using 
the  HHDD  correctly  identified  the  inserted  fault  more  often 
and  performed  the  task  faster  than  technicians  using  the 
PMA.  Furthermore,  for  the  Engine  Bay  Inspection,  HMDD- 
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equipped  technicians  identified  more  discrepancies  than  PMA- 
equipped  technicians. 

In  the  opinion  of  the  researchers,  the  inprovenent  in 
technician  performance  was  related  to  easy  accessibility  to 
technical  data  inherent  in  the  HMDD.  By  having  the  data 
available  within  the  technicians'  field  of  view,  they  were 
more  likely  to  read  the  information  in  smaller  segments  and 
at  a  higher  sampling  rate.  In  contrast,  it  was  more 
difficult  for  a  technician  to  obtain  technical  data 
displayed  by  a  hand-held  computer  like  the  PMA  while 
performing  a  maintenance  activity.  As  a  result,  technicians 
equipped  with  the  PMA  were  less  likely  to  view  the 
information  as  often  as  the  HMDD-equipped  technicians. 
Technicians  who  frequently  view  the  data  found  it  easier  to 
determine  what  step  they  were  on,  when  the  step  was 
completed,  and  what  action  was  required  next. 

Experience  Level 

Technician  experience  level  affected  the  performance 
measures  as  well.  Experienced  technicians  correctly 
identified  the  faults  more  often  than  the  less-experienced 
technicians.  In  the  case  of  the  Fault  Isolation  Test,  the 
experienced  technicians  completed  the  tests  faster  than 
their  less-experienced  counterparts  and  exhibited  a  higher 
success  rate  in  identifying  the  inserted  fault.  For  the 
Engine  Bay  Inspection,  the  experienced  technicians  found 
more  discrepancies  and  found  the  inserted  discrepancy  more 
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often  than  their  less-experienced  counterparts.  In  the 
process,  the  experienced  technicians  took  longer  to  complete 
the  inspection  than  their  less-experienced  counterparts. 

The  researchers  believe  that  experienced  technicians 
were  more  familiar  with  the  procedures  examined.  Therefore, 
the  experienced  technicians  were  able  to  comprehend  the 
indicators  provided  to  them  during  a  maintenance  activity 
to  a  fuller  extent. 

Type  of  Task 

The  type  of  task  performed  was  a  major  factor  in  the 
performance  measurements  of  this  study.  In  reviewing  the 
completion  times,  it  was  apparent  that  the  two  tasks 
required  different  amounts  of  time  to  perform  the 
activities.  The  number  of  faults  found  also  differed  for 
each  task.  The  very  nature  of  the  Fault  Isolation  Test 
limited  the  number  of  faults  found  to  one.  The  Engine  Bay 
Inspection,  on  the  other  hand,  had  over  20  possible  faults 
that  could  have  been  identified.  However,  it  is  the  task's 
influence  on  the  relationship  of  the  two  performance 
measures  that  is  significant. 

Task  Influence  on  Performance  Measures 

In  the  Fault  Isolation  Test,  the  two  performance 
measures  were  found  to  be  congruent.  Technicians  who 
performed  the  task  quicker  also  found  the  inserted  fault 
more  often.  However,  as  discussed  in  Chapter  4,  in  the 
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section  on  the  "Faults  and  Completion  Times  Interaction," 
the  completion  times  and  the  number  of  faults  found  in  the 
Engine  Bay  Inspection  task  provided  diverging  indications  of 
technician  performance.  Shorter  completion  times  were  in 
conflict  with  the  number  of  faults  found.  In  general,  the 
technicians  who  found  the  greater  number  of  faults  were  the 
same  technicians  who  took  longer  to  complete  the  task.  In 
the  case  of  the  Engine  Bay  Inspection,  a  longer  completion 
time  was  an  indication  of  more  thorough  visual  inspection 
and  improved  performance.  This  would  substantiate  the  need 
for  two  separate  types  of  measures  (in  other  words,  both 
efficiency  and  effectiveness)  in  evaluating  technician 
performance. 

Experience /Task  Interaction 

In  addition  to  the  main  effects,  experimental  results 
suggested  that  interaction  effects  influenced  both 
performance  measures.  As  mentioned  in  Chapter  3,  "the  main 
effect  of  a  variable  should  be  individually  interpreted  only 
if  there  is  no  evidence  that  the  variable  interacts  with 
other  variables"  (Box,  178:317).  In  evaluating  the  task 
completion  times,  the  experience/ task  interaction  effect  was 
found  significant.  When  performing  the  Fault  Isolation 
Test,  experienced  technicians  completed  the  task  faster  than 
the  less-experienced  technicians  on  average.  Conversely, 
for  the  Engine  Bay  Inspection  task  the  trend  was  reversed. 
The  less-experienced  technicians  took  less  time  performing 
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the  task  than  did  the  experienced  technicians  on  average. 
Thus,  it  would  be  inappropriate  to  focus  solely  on  the 
effect  of  experience  without  also  taking  into  consideration 
the  type  of  task  being  performed. 

Experience /Display /Task  Interaction 

The  experience /display  interaction  affected  the  Fault 
Isolation  Test's  completion  times  differently  than  the 
Engine  Bay  Inspection's  number  of  faults  found.  This 
dependence  on  the  task  is  referred  to  as  a  three-factor 
interaction. 

For  the  Fault  Isolation  Test,  less-experienced 
technicians'  completion  times  showed  more  improvement  with 
the  use  of  the  HMDD  than  for  the  experienced  technicians. 

It  is  the  researchers'  assessment  that  the  nature  of  the 
Fault  Isolation  Test  restricted  the  experienced  technicians' 
range  of  improvement.  A  possible  explanation  of  the  task's 
influence  on  this  interaction  effect  is  that  the  Fault 
Isolation  Test  was  a  step-by-step  procedure  that  required 
precise  actions  to  be  performed.  Better  performance  in  this 
task  was  indicated  by  shorter  completion  times  and  correct 
identification  of  the  inserted  fault.  Since  experienced 
technicians  tend  to  operate  the  aircraft's  systems  quicker 
regardless  of  the  display  device  used,  there  was  little  room 
for  improving  their  performance  by  using  the  HMDD. 

Therefore,  the  possible  range  of  performance  improvement  by 
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an  experienced  technician  was  smaller  than  for  the  less- 
experienced  technician. 

For  the  Engine  Bay  Inspection  task,  the  HMDD  enhanced 
the  experienced  technicians'  performance  more  so  than  the 
less-experienced  technicians'  performance.  The  Engine  Bay 
Inspection  was  a  more  broadly  defined  task.  The  technical 
instructions  for  this  task  advised  technicians  to  inspect 
specific  areas  and  to  look  for  certain  conditions.  From 
these  conditions,  the  technician  was  required  to  assess  the 
equipment's  air  worthiness.  In  contrast  to  the  Fault 
Isolation  Test,  the  experienced  technicians'  range  of 
performance  improvement  was  greater.  In  addition  to  having 
a  greater  range  of  performance  improvement,  the  Engine  Bay 
Inspection  task  appeared  to  provide  the  experienced 
technicians  an  advantage.  Because  the  technician  using  the 
HMDD  reads  the  information  more  frequently  and  in  smaller 
sections,  he  or  she  receives  more  cues  or  flags  as  to  what 
to  look  for.  The  increased  frequency  of  cues  combined  with 
an  experienced  technician's  larger  knowledge  base  resulted 
in  the  experienced  HMDD-equipped  technicians  finding  the 
most  faults  as  a  group. 

Participants*  Perceptions 

Technicians  considered  both  the  HMDD  and  the  PMA  as 
suitable  displays  for  flightline  maintenance.  The 
technicians  who  used  the  PMA  generally  rated  its  performance 
higher  in  satisfaction  than  technicians  using  the  HMDD,  with 
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the  only  exception  relating  to  computer  response  time.  The 
PMA  used  in  this  experiment  had  screen  retrieval  times  much 
slower  than  the  HMDD.  AL  had  a  PMA  with  essentially 
equivalent  screen  retrieval  times,  but  it  was  not  available 
for  use  in  this  experiment.  When  asked  what  they  disliked 
about  the  devices,  the  majority  of  the  participants 
responded  with  recommendations  to  improve  system  hardware  or 
software.  The  majority  felt  that  each  device  would  benefit 
technician  performance. 

When  asked  about  visual  symptoms  associated  with  the 
use  of  the  HMDD  in  a  dynamic  environment,  some  of  the 
technicians  seemed  to  experience  a  slight  degree  of  eye 
strain.  The  eye  strain  appeared  to  result  from  an 
unfamiliarity  with  switching  attention  from  the  HMDD  to  the 
surrounding  environment  and  back.  In  the  interview 
responses,  several  technicians  indicated  that  it  took  a 
little  practice  to  become  accustomed  to  using  the  HMDD. 

While  technicians  tended  to  squint  more  during  the  initial 
part  of  the  tasks  performed,  most  technicians  were  viewing 
the  HMDD  without  squinting  by  the  end  of  the  trial. 

The  researchers  assessed  the  conditions  associated  with 
the  HMDD  tendency  to  experience  screen  fade  in  bright  light 
conditions.  They  found  that  the  HMDD  screen  was  unreadable 
when  the  wearer  faced  directly  into  the  sun.  The  problem 
appeared  to  be  the  sunlight's  reflection  on  the  wearer's 
face.  A  polarized  tinted  filter,  placed  in  front  of  the 
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screen,  reduced  the  reflection  enough  so  that  the  wearer 
could  read  the  screen. 

gmwnery 

The  purpose  of  this  study  was  to  determine  if  there 
were  any  additional  benefits  in  using  a  monocular 
head-mounted  display  device  as  opposed  to  a  hand-held  flat- 
screen  display  device  to  present  maintenance  technical 
orders  to  technicians  performing  flightline  tasks.  By  using 
the  HMDD,  technicians  were  provided  easy  access  to  technical 
instructions  during  the  performance  of  a  flightline 
maintenance  activity.  The  results  indicate  that  technicians 
performing  tasks  with  data  displayed  on  the  HMDD  generally 
performed  better  than  their  PMA-eguipped  counterparts. 
Experience  was  also  an  influencing  factor.  HMDD-equipped 
experienced  technicians  performing  the  Engine  Bay  Inspection 
were  able  to  locate  more  discrepancies  than  any  other  group 
of  technicians.  The  test  data  appears  to  favor  the  HMDD 
over  the  PMA.  The  degree  to  which  these  test  results  apply 
to  other  systems  and  circumstances  remains  to  be  seen. 

The  next  chapter  addresses  the  value  of  these  findings 
with  respect  to  the  research  question.  In  addition, 
insights  on  some  possible  improvements  to  the  display 
devices  and  potential  areas  for  further  research  are 
provided . 


VI.  Conclusions  and  Recommendations 


QYsrvigy 

After  reviewing  the  experimental  results  and  the 
discussions  of  the  factors  found  to  be  influencing  the 
experimental  measures,  it  is  appropriate  to  examine  them  in 
light  of  the  overall  research  question.  This  chapter  will 
investigate  this  question  and  will  discuss  some  possible 
improvements  to  the  display  devices  used  and  some  areas  that 
could  benefit  from  further  research. 

Conclusions 

Depending  on  the  task  and  the  performance  measure 
considered,  the  display  type  significantly  affects  the 
performance  of  technicians  performing  flightline  maintenance 
activities.  For  the  Fault  Isolation  Test,  technicians 
completed  the  task  faster  with  the  HNDD  than  with  the  PMA. 
For  the  Engine  Bay  Inspection,  technicians  found  more 
discrepancies  using  the  HMDD  than  with  the  PMA. 

Experience  level  also  had  a  significant  effect  on  the 
technician's  performance  of  flightline  maintenance 
activities,  again  depending  on  the  task  and  performance 
measure  considered.  For  the  Fault  Isolation  Test, 
experienced  technicians  performed  the  task  faster  and 
exhibited  a  higher  success  rate  in  identifying  the  inserted 
fault.  For  the  Engine  Bay  Inspection,  experienced 
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technicians  using  the  HHDD  outperformed  all  other  groups 
tested  by  finding  more  faults.  These  findings  are  congruent 
with  Masguelier's  results  which  found  experience  level  to  be 
significant.  Future  studies  in  this  area  might  consider 
blocking  on  experience  level  in  their  initial  design. 

Armed  with  these  experimental  findings,  the  researchers 
are  confident  that  the  Air  Force  should  consider  employing 
HMDD  systems  for  displaying  technical  data  in  a  flightline 
maintenance  environment.  This  study  has  shown  that  there 
are  some  tasks,  specifically  complex  procedural  tasks  and 
inspection  tasks,  that  may  benefit  from  use  of  an  HMDD. 
However,  the  use  of  an  HMDD  cannot  be  considered  suitable 
for  every  maintenance  action  performed  on  a  flightline. 

As  in  most  studies,  the  information  presented  here 
comes  with  some  general  caveats.  Tests  conducted  at  a 
national  guard  unit  may  have  some  associated  biases  that 
limit  generalization  to  active  duty  units.  Due  to  the 
exploratory  nature  of  this  study,  time  restrictions,  and 
equipment  and  technician  pool  limitations,  the  data 
collected  represents  a  small  cross-section  of  the  type  of 
tasks,  equipment,  and  organizations  that  could  be  evaluated. 

Recommendations 

One  of  the  added  benefits  of  exploratory  research  is 
that  deficiencies  can  be  identified  early  in  the  systems 
development.  The  HMDD  and  PMA  units  used  during  the 
experiment  were  one-of-a-kind  prototypes.  Use  by  several 
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operational  technicians  in  a  realistic  maintenance 
environment  provided  an  early  opportunity  to  suggest 
possible  improvements  to  the  systems.  This  section  is 
divided  into  four  separate  areas  for  discussion:  possible 
improvements  to  HMOD  system,  possible  improvements  to  the 
PMA  system,  possible  improvements  common  to  the  HMDD  and 
PHA,  and  recommendations  for  follow-up  experimentation. 

Possible  Improvements  to  the  HMDD.  Adjustment  of  the 
headset,  which  was  necessary  to  view  the  entire  screen,  was 
a  problem  for  several  technicians.  Difficulties  were 
encountered  when  first  putting  on  the  ensemble  and  then 
positioning  the  eyepiece  (or  CRT)  to  where  the  technician 
could  view  the  entire  screen.  Primarily,  the  weight  of  the 
screen  and  cable  seem  to  cause  the  problem.  Some 
technicians  commented:  "It  tended  to  slip  a  bit  when  I 
tilted  my  head  back,"  and  "It  seemed  that  the  headset  would 
start  to  fall  off  when  I  looked  up."  To  fit  the  headband, 
the  technician  first  positioned  the  eyepiece  to  where  he 
could  see  the  entire  screen  of  text.  Then,  a  locking  knob 
on  the  side  of  the  headset  was  tightened  to  secure  the 
eyepiece's  position  in  front  of  the  technicians'  eye. 
However,  as  soon  as  the  technician  removed  his  hand  from  the 
eyepiece,  it  would  fall  approximately  one  quarter  inch  (see 
Figure  16) .  Because  the  headset  lacked  a  method  of  making 
minor  adjustments,  the  best  way  to  fine-tune  the  position  of 
the  eyepiece  was  by  adjusting  the  position  of  the  entire 
headband.  Attempts  to  reposition  the  CRT  and  resecure  the 
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Figure  16.  Adjusting  the  HMDD's  CRT 

locking  knob  did  not  work  as  well.  A  method  of  making  minor 
adjustments  would  be  preferred  over  the  all-or-nothing 
method  provided  by  the  locking  knob.  A  possible  solution 
could  be  a  threaded  arrangement  that  would  allow  the  wearer 
to  incrementally  change  the  elevation  and  azimuth  of  the 
CRT. 

The  CRT  also  moved  during  both  tests.  The  technician 
would  have  to  stop  and  adjust  the  position  of  the  headband 
to  bring  the  entire  screen  back  into  viewing  position. 
Unfortunately,  wearing  the  headband  tighter  did  not  solve 
the  problem.  One  comment  was:  "when  I  got  it  tight  enough 
that  it  would  not  move,  it  became  uncomfortable  to  wear 
after  awhile."  More  effective  methods  of  securing  the 
eyepiece  in  position  should  be  investigated. 
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The  subject  of  screen  fading  was  discussed  at  length  in 
the  Screen  Fading  section  of  chapter  4.  The  results  of  that 
investigation  suggested  that  placing  a  tinted  polarized 
filter  on  the  front  of  the  screen  nay  alleviate  nost  of  the 
wearer's  problens  with  operating  the  HMDD  in  bright  lighting 
conditions. 

The  HMDD  system  generally  received  favorable  comments. 
However,  at  least  three  technicians  performing  the  Fault 
Isolation  Test  commented  on  the  discomfort  caused  by  the 
components  in  the  back  of  the  vest.  Some  technicians 
commented;  "The  battery  pack  and  stuff  on  the  back  got  in 
the  way"  and  "Equipment  on  the  back  is  uncomfortable." 
Equipment  is  also  located  on  the  front  of  the  vest,  and  in 
certain  bending  and  twisting  motions,  the  equipment  might 
become  uncomfortable.  Repositioning  of  the  components, 
miniaturization,  and  padding  are  all  possible  solutions  to 
this  problem.  The  most  common  complaint  about  the  vest  was 
heat  buildup  from  physical  exertion  while  performing  the 
experiments  and  functioning  of  the  electronic  components. 

One  suggestion  was  to  make  the  vest  out  of  mesh  material. 

Possible  Improvements  to  the  PMA.  Negative  comments  on 
the  PMA  focused  on  its  bulk,  weight,  and  screen  durability. 
With  the  advances  seen  in  portable  computers,  improvements 
in  these  areas  should  be  possible.  One  technician 
recommended  that  a  stand  be  incorporated  into  the  system  so 
the  user  could  tilt  the  display.  A  stand  could  reduce  glare 
on  the  screen  and  make  it  possible  for  a  technician 
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performing  a  task  to  read  the  screen  without  having  to  stand 
over  the  PMA.  A  possible  solution  to  technician  concern 
over  damaging  the  screen  would  be  to  mount  a  plexiglass 
shield  on  it.  The  shield  would  have  to  be  replaced  after 
its  translucency  dulls;  however,  the  cost  would  be  less  than 
replacing  the  screen. 

Possible  Improvements  Cororoon  to  the  HMDD  and  the  PMA. 
Batteries  continue  to  be  a  hinderance  to  the  use  of  portable 
electronic  display  devices.  Although  the  PMA  battery  had  a 
longer  life  than  the  HMDD,  both  systems  required  frequent 
recharging.  Several  technicians  voiced  concern  over  the 
reliability  of  these  systems  to  display  information  when 
needed.  Thus,  both  systems  could  benefit  from  a  power 
management  mode  that  would  pause  the  system  and  power  down 
the  display.  Reactivation  of  the  screen  would  have  to  be 
easily  and  rapidly  performed  to  avoid  hindering  the 
technician.  Several  times  during  testing,  the  display 
devices  had  to  be  powered  down  due  to  insufficient  battery 
life  or  system  lockup.  In  each  case,  the  technical  data  had 
to  be  cycled  through  screen  by  screen,  to  the  point  where 
the  system  powered  down.  If  the  trial's  procedures  had  been 
lengthier,  returning  to  a  point  where  the  technician  could 
have  resumed  would  have  required  a  significant  amount  of 
effort.  Moreover,  if  such  a  power  down  were  to  occur  on  a 
complex  maintenance  activity,  there  would  be  increased  risk 
that  the  technician  may  not  return  to  the  correct  place  in 
the  technical  instructions.  Therefore,  it  would  also  be 
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beneficial  to  have  an  electronic  bookmark  that  allows  the 
technician  to  return  to  the  last  screen  viewed  before  system 
shutdown. 

Further  Experimentation.  It  is  the  researchers' 
opinion  that  further  work  is  warranted.  Past  research  has 
dealt  with  the  use  of  the  HMDD  in  a  very  dynamic  (flying) 
environment  and  in  a  shop  bench-level  repair  environment 
(Chapter  2).  The  current  research  focused  on  exploring  the 
effect  of  display  devices  on  two  aircraft  maintenance 
activities  in  a  flightline  environment.  Additional 
maintenance  tasks  should  be  evaluated  using  the  two  display 
devices.  Testing  should  include  activities  requiring  more 
than  one  technician,  activities  requiring  in-depth  trouble¬ 
shooting  analysis,  activities  requiring  fast  response 
turnaround,  and  activities  requiring  schematics.  One 
technician  said:  "Schematics  would  be  useful  for  chasing 
down  wires  during  troubleshooting.  It  would  be  easier  than 
the  fold  out  charts  the  T.O.s  have." 

Further  testing  should  consider  using  a  more  complex 
testbed.  The  A-7D  electronics  equipment  does  not  have  the 
complexity  of  a  more  modern  aircraft,  such  as  an  F-15  or  F- 
16.  It  may  be  the  case  that  the  display  device  effect  on 
the  technicians'  performance  would  be  more  pronounced  on  a 
more  complex  test  bed.  As  stated  by  one  of  the  technicians: 
"The  maintenance  tasks  involved  in  those  aircraft  would 
require  a  larger  amount  of  data  in  different  T.O.  volumes." 
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There  is  also  the  human  factors  issue  of  HMDD 
suitability  in  an  Nuclear-Biological-Chemical  environment. 
Some  technicians  stated  that  performing  maintenance 
activities  while  wearing  chemical  gear  might  be  a  problem. 
For  instance,  the  eye-piece  (as  currently  designed)  would 
have  to  be  worn  outside  the  suit's  plexiglass  faceshield, 
which  would  affect  the  technician's  ability  to  view  the 
display  and  accomplish  necessary  maintenance  tasks. 
Therefore,  the  compatibility  of  the  HMDD  with  protective 
clothing  warrants  further  study. 

Two  factors  should  be  considered  in  developing  any 
future  experiment's  strategy:  (1)  use  of  more  than  one 
performance  measure,  and  (2)  the  use  of  factorial  design  (as 
opposed  to  the  one-variable-at-a-time  method) .  This  study 
found  that  due  to  the  complexity  of  technician  performance, 
one  measure  could  provide  misleading  results.  Two  or  more 
measures  provide  a  more  complete  picture  of  the  behavior 
being  studied.  Factorial  analysis  is  an  effective  tool  in 
evaluating  hviman-machine  interactions,  since  it  provides  the 
ability  to  analytically  evaluate  the  complex 
interrelationships  likely  to  be  encountered. 
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Appendix  A;  HMDD  and  PMA  Hardware 


HMDD  Ensemble  Hardware 

The  VGA  HMDD  used  in  the  current  research  was 
manufactured  by  Imaging  &  Sensing  Technology.  It  used  a 
miniature  CRT  with  a  display  format  of  640  x  480  pixels  on  a 
monochrome  (b/w)  screen.  The  CRT  had  a  picture  size  of  6mm 
X  8mm,  cylinder  size  of  20mm  in  diameter  and  80mm  long,  and 
a  weight  of  50g.  A  focusing  mechanism  weighing  60g  was 
attached  to  the  bottom  of  the  CRT  case.  This  assembly, 
including  the  attached  video  cable,  was  mounted  on  an 
adjustable  surgeon's  headband.  The  video  cable  connected 
the  CRT  to  the  drive  electronics.  Input  supplied  to  the  CRT 
from  the  drive  electronics  was  at  a  vertical  scan  rate  of 
59-90  Hz  non-interlaced,  horizontal  scan  rate  of  31.5  Hz, 
and  the  video  was  at  40  Hz  analog.  The  drive  components 
were  mounted  in  a  vest  worn  by  the  test  subject.  The  power 
supply  batteries  were  also  contained  in  vest  pockets.  The 
two  batteries  were  12V,  lead-acid.  Quasar  products.  The 
computer  drive  was  a  386  with  16MB  of  RAM  and  a  20MB  flash 
drive.  Power  was  connected  with  an  on/off  toggle  switch 
located  in  an  upper  left  front  vest  pocket.  It  was  detented 
to  prevent  it  from  being  inadvertently  moved  from  one 
position  to  the  other.  The  input  devices  used  were  two 
different  types.  The  first  was  an  aircraft  type  control 
stick.  Two  buttons  were  located  on  the  top,  one  button 
retrieved  the  next  screen  of  information,  the  other  button 
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retrieved  the  previous  screen.  The  other  input  device  was  a 
telephone  keypad  arrangement  with  two  buttons  performing  the 
same  function  as  with  the  control  stick.  The  configuration 
of  the  HMDD  system  is  in  Figures  17  and  18  shown  below. 


Figure  17.  HMDD  System  Configuration  (Back  View) 


The  PMA  used  was  assembled  by  AL.  It  used  an  Ovionics 
transflective  LCD  screen  with  a  display  format  of  640  x480 
pixels  on  a  solid  monochrome  screen.  Picture  size  was  6"  x 
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Figur«  18.  HMDD  System  Configuration  (Front  View) 


8”.  The  drive  electronics  were  from  a  Motorola  68020  based 
hybrid  with  6MB  of  RAM  and  4MB  of  flash  memory.  The 
graphics  coprocessor  was  made  by  Intel.  Power  supply  was 
from  a  17V  single  silver  cell  battery  pack  (11  x  1.5).  All 
the  components,  except  for  the  battery,  were  enclosed  in  a 
carbon  fiber  case  measuring  10.5"  x  9.5"  x  3.0".  The  entire 
assembly  weighed  approximately  7  pounds.  Input  to  the 
computer  was  from  a  keyboard  mounted  on  the  top  of  the  case. 
For  the  experiment,  only  two  keys  were  needed.  One  key 
retrieved  the  next  screen  of  text  and  one  other  key 


retrieved  the  previous  screen.  This  arrangement  was 
consistent  for  both  the  HMDD  and  the  PMA.  The  PMA 
configuration  is  shown  in  Figi  re  19. 


Figure  19.  PMA  System  Configuration 


Appendix  B;  User  Evaluation  Questionnaire 


Please  answer  the  following  questions  based  on  your 
participation  in  the  evaluation.  The  questionnaire  is 
divided  into  three  sections  with  questions  on  visual  aspects 
related  to  the  use  of  each  display,  cpiestions  on  the 
physical  device,  and  questions  on  the  information  presented. 
Please  read  the  questions  carefully  and  put  a  circle  around 
the  appropriate  response. 

I.  Questions  on  the  visual  Aspects  of  the  Displays 


HMDD 

1.  Capability  of  switching  your  attention  from  the  display 
to  your  work. 

Highly 

Outstanding  Satisfactory  Satisfactory  Marginal  Unsatisfactory 


A  B 

C 

D 

E 

2.  Eye  Strain. 

Highly 

Outstanding  Satisfactory 

1  1 

Satisfactory 

1 

Marginal 

1 

Unsatisfactory 

1 

1  1 

A  B 

1 

c 

1 

D 

1 

E 
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3 .  Blurring 


Blshly 

Outstandins  Satisfactory  Satlafactory  Marslnal  Unsatisfactory 


A 

B 

C 

D 

E 

4.  Focusing. 

Outstanding 

Highly 

Satisfactory 

Satisfactory 

Marginal 

Unsatisfactory 

A 

B 

C 

D 

E 

.  Head 

Piece. 

Outstanding 

Highly 

Satisfactory 

Satisfactory 

Marginal 

Unsatisfactory 

A  B 

C 

D 

E 

.  Glare  on  the  screen. 

Highly 

Outstanding  Satisfactory 

Satisfactory 

Marginal 

Unsatisfactory 

A 


B 


C 


D 


E 


7.  Readability  of  all  the  information  on  the  screen. 

Highly 

Outstanding  Satisfactory  Satisfactory  Marginal  Unsatisfactory 


A  B 

C 

D 

E 

.  After  images. 

Highly 

Outstanding  Satisfactory 

Satisfactory 

Marginal 

Unsatisfactory 

A 


B 


C 


D 


E 
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9.  Green  spots  after  using  the  HMDD. 

Hishly 

Outctandins  Satisfactory  Satisfactory  Marcinal  Unsatisfactory 


A 


B 


C 


D 


E 


10.  Headaches  from  using  the  HMDD. 

BiBhly 

Outstanding  Satisfactory  Satisfactory  Marginal.  Unsatisfactory 


A 


B 


C 


D 


E 


11.  Contrast  between  the  information  displayed  and  the 
background . 

Bighly 

Outstanding  Satisfactory  Satisfactory  Marginal  Unsatisfactory 


A  B  C  D  E 


12.  Brightness  of  the  display. 


Bighly 

Outstanding 

1 

Satisfactory 

..  1  _ 

Satisfactory 

1 

Marginal 

1 

Unsatisfactory 

1 

1 

A 

1 

B 

t 

c 

1 

D 

t 

£ 
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PNA 


1.  Glare  on  the  screen. 

Blghlr 

Outstandlnc  Satlsfaetoty  Satlafaetory 

ABC 


Marginal 


D 


Unaatiafaetory 


t 


2.  Readability  of  all  the  information  on  the  screen. 

Highly 

Outstanding  Satlstaetory  Satisfactory  Marginal  Unsatisfactory 


A 


B 


C 


D 


E 


3.  Contrast  between  the 
background . 

information  displayed 

and  the 

Outstanding 

1 

Highly 

Satisfactory 

1 

Satisfactory 

1 

Marginal 

1 

Unsatisfactory 

1 

1 

A 

1 

B 

1 

C 

1 

D 

1 

E 

4.  Black  text  on  the  white  background. 


Outstanding 

1 

Highly 

Satisfactory 

1 

Satisfactory 

1 

Marginal 

1 

Unsatiafactor/ 

1 

1 

A 

1 

B 

1 

C 

1 

D 

1 

E 

5.  Brightness  of  the  display. 


Highly 

Outstanding 

Satisfactory 

Satisfactory 

Marginal 

Unsatisfactory 

A 


B 


C 


D 


E 
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KMDD 


1.  Focusing  mechanism. 

HiChly 

Outstanding  Satlsfaetexy  Satlsfaetexy  Harginal  Unsatisfactory 

A _ _ _ t _ I _ I _ L 


1  I  I  I 

A  B  CUE 
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6.  Computer  response  time 


Highly 

Outstanding 

J _ 

Satiafactory 

_ 1 _ 

Satisfactory 

1 

Marginal 

1 

Unaatisfaetory 

1 

1 

A 

1 

B 

1 

C 

- 1 - 

D 

- r 

E 

PMA 

1 .  Keyboard . 

Highly 

Outstanding  Satlsfaetoxy  Satiafsctoty  (terginal  Onsatisfaetory 


A 


B 


C 


D 


E 


2.  Adequacy  of  the  screen  size  for  displaying  information. 

Highly 

Outstanding  Satisfactory  Satisfactory  Marginal  Unsatisfactory 


A  B  C  D  E 

3.  Computer  response  time. 

Highly 

Outstanding  Satisfactory  Satisfactory  Marginal  Unsatisfactory 


A 


B 


C 


D 


E 


BNDD 


1.  Readability  of  the  text. 


Outstanding 

1 

Highly 

Satisfactory 

1 

Satisfactory 

1 

Marginal 

1 

Unsatisfactory 

1 

1 

A 

1 

B 

1 

C 

1 

D 

1 

E 

• 

2.  Spacing  of  information  on  the  display  screen  (lack  of 
clutter,  .etc.) 

Hlshly 

Outstanding  Satisfactory  Satisfactory  Marginal  Unsatisfactory 


A 


B 


C 


D 


PMA 

1.  Readability  of  the  text. 

Highly 

Outstanding  Satisfactory  Satisfactory  Marginal  Unsatisfactory 


A 


B 


C 


D 


E 


2.  Spacing  of  information  on  the  display  screen  (lack  of 
clutter,  etc.) 


Outstanding 


Highly 

Satisfactory 


Satisfactory 


Marginal 


Unsatisfactory 


Appendix  C:  Structured  Interview 


The  following  (questions  were  asked  of  technicians  using  the 
PNA. 


1.  Is  the  information  on  the  screen  easy  to  read? 

2.  Can  you  give  some  examples  of  other  activities  that  would 
be  enhanced  by  using  this  device. 

3.  What  did  you  not  like  about  the  device? 

4.  Were  there  any  Visual  problems  with  reading  the  screen 
(problems  with  glare)? 

5.  Was  the  task  you  performed  representative  of  maintenance 
activities  you  are  asked  to  routinely  accomplish? 

6.  Looking  at  the  manner  in  which  we  conducted  the 
experiment,  including  the  prebrief,  instructions,  eye 
examination,  experiment,  survey,  and  interview,  would  you 
like  to  see  any  changes  made?  Was  the  information  adequate? 

7.  Do  you  have  any  other  questions  or  comments? 

The  following  questions  were  asked  of  the  technicians  using 
the  HMDD. 

1.  Is  the  information  on  the  screen  easy  to  read? 

2.  Can  you  give  some  examples  of  other  activities  that  would 
benefit  from  using  this  device? 

3.  What  did  you  not  like  about  the  device? 

4.  Were  there  any  Visual  problems  with  reading  the  screen 
(problems  with  glare)? 

5.  Did  you  suffer  any  headaches  or  eye  strain  from  using  the 
device? 

6.  Did  you  have  any  problems  with  the  headband? 

7.  Did  you  have  any  problem  with  the  vest? 

8.  Was  the  task  you  performed  representative  of  maintenance 
activities  you  are  asked  to  routinely  accomplish? 

9.  Looking  at  how  we  conducted  the  experiment,  including  the 
prebrief,  instructions,  eye  examination,  experiment,  survey, 
and  interview,  would  you  like  to  see  any  changes  made?  Was 
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the  information  adequate? 

10.  Do  you  have  any  other  questions  or  comments? 
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Appendix  D;  Personal  Background 


Subject  ID 


1. 

Check  one:  Full-Time 

Traditional 

Guardsman 

Guardsman 

2. 

Time  in  Service: 

(yrs/mons) 

3. 

Paygrade/Rank : 

4. 

Current  Specialty:  (Job  Title/AFSC) 

5. 

Prior  Work  Experience: 

(1)  (2) 

(3) 

Work  History 

Number  of  Years 

6. 

Education: 

Education  (Highest  Grade  Completed) 
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Appendix  E;  Engine  Bav  Inspection  and  Radar  Fault  Isolation 
Test  Technical  Data 


Engine  Bey  Inspection  Technioel  Dete: 

1.  Fuel  system. 

A.  All  fuel  and  vent  lines  for  dents,  nicks, 
scratches,  chafing,  cracked  B-nuts  or  sleeves,  loose  wig<^ins 
fitting,  broken  or  improper  size  clamps  (T.O. 
1A-7D-2-6/1A-7K-2-6,  T.O.  l-lA-8  and  T.O.  44H3-1-3) . 

B.  Hoses  and  tubing  for  collapsing,  cracks,  leakage, 
and  insecurity  (T.O.  1A-7D-2-6/1A-7K-2-6) . 

C.  Motive  flow  disconnect  coupling  (215-53424-1,  -3) 
for  missing  or  broken  aluminum  locking  dogs  (T.O. 
1A-7D-2-5/T.O.  1A-7K-2-5) . 

D.  Fuel  transfer  selector  valves  (3)  for  corrosion  and 
condition  (T.O. 1A-7D-2-6/T.O.  1A-7K-2-6) . 

2.  UHT  system. 

A.  UHT  backup  torque  tube  bolts  (2)  for  broken 
lockwlre. 


B.  Exposed  areas  of  interconnect  controlex  for 
cleanliness  (do  not  lubricate) . 

C.  Controlex  cable  for  binding  in  clamps,  and  proper 
size  clamps.  Disconnect  controlex  rod  ends  from  bellcrank 
links.  Disconnect  bellcrank  from  right  load-limiting  link 
to  provide  access  for  spring  scale.  Determine  force 
required  to  move  the  sliding  part  of  controlex  through  full 
travel  in  both  directions.  Force  should  not  exceed  4.0 
pounds.  Notify  test  conductors  of  results.  Connect 
bellcrank  to  load  limiting  link.  Connect  controlex  rod  ends 
to  bellcrank  links. 

D.  Funk  springs  struts  (2)  for  broken  lockwire  and 
insecurity  of  mounting. 

E.  Inspect  left  and  right  UHT  backup  connecting  links 
(P/N  216-48101-2)  for  looseness  and  insecurity. 

3.  Fire  detection  system. 

A.  Sensing  cable  elements  for  insecurity  of  mounting, 
crimping,  distortion,  and  corrosion. 

4.  Electrical  system  components  and  wiring. 
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A.  All  visible  electrical  wiring  and  connectors  for 
fraying,  chafing,  loose  or  missing  parts,  fluid  saturation, 
and  serviceability. 

B.  Battery  connectors  for  condition  and  corrosion. 

5 .  The  end 


NOTE 

THIS  IS  THE  LAST  SCREEN  OF  THE  TEST.  IF  YOU 
SELECT  BACKUP  YOU  CAN  REVIEW  THE  ABOVE  STEPS.  IF 
YOU  ARE  FINISHED  THEN  SELECT  NEXT. 

************************************************************ 
Radar  Fault  Isolation  Test: 

1.  The  radar  set  is  considered  to  be  operating  normally 
when  proper  responses  as  specified  in  the  following 
procedures  are  obtained;  however,  a  complete  evaluation  of 
the  radar  system  operation  requires  additional  testing.  If 
a  flight  discrepancy  cannot  be  duplicated  during  performance 
of  the  operational  checkout,  perform  appropriate 
troubleshooting  test  or  tests  (paragraphs  4-46  and  4-69) . 

WARNING 

Personnel  and  combustible  materials  must  be  kept 
out  of  microwave  radiation  hazardous  area  when 
radar  is  transmitting.  Refer  to  maintenance 
precautions  for  hazardous  distance  information. 

To  prevent  possible  serious  injury  or  death  to 
personnel,  insure  that  requirements  of  T.O. 
llA-1-33  are  complied  with  before  doing  any  of  the 
following  maintenance  procedures. 

CAUTION 

When  working  on  or  near  the  antenna  assembly,  be 
careful  not  to  damage  antenna  assembly  components. 

Do  not  use  waveguide  sections  or  reflector 
assembly  for  handholds. 

2.  Connect  external  electrical  power  (paragraph  3-1). 

3.  Ensure  RF  absorbing  curtain  is  in  front  of  antenna. 

4.  Ensure  weight-off -gear  bypass  switch  is  in  NORMAL. 

5.  Ensure  power  switch  on  radar  set  control  is  in  ON. 

CAUTION 
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Before  energizing  radar  set,  ensure  antenna 
boresight  lockpins  (elevation  and  azimuth)  are 
disengaged  to  avoid  damage  to  stabilization 
servos . 


NOTE 

As  an  alternate  procedure,  a  MIL-T-26772  nitrogen 
service  trailer  may  be  used  to  maintain  waveguide 
pressurization  at  14.5  (+/*  0.5)  psi. 

NOTE 

At  facilities  where  field  elevation  is  4000  feet 
above  sea  level  or  higher  a  jumper  plug  may  be 
temporarily  installed  on  mount  harness  connector 
2P5  to  bypass  the  transmitter  pressure  switch. 

This  jumper  plug  should  be  installed  prior  to 
applying  power  to  radar  set  and  should  not  be 
removed  until  after  removing  power  from  radar  set. 

NOTE 

If  the  radar  set  automatically  shuts  down, 
overheating  is  indicated.  If  power 
supply-programmer  or  transmitter  components 
repeatedly  fail,  the  centrifugal  fan  should  be 
checked  for  operation  or  damage,  and  fan  screens 
should  be  checked  for  clogging. 

NOTE 

A  number  or  numbers  enclosed  in  parentheses  at  the 
end  of  a  step  in  the  following  checkout  is 
reference  to  a  corresponding  number  in 
troubleshooting  table  4-3. 

6.  Place  radar  set  power  switch  in  STBY.  Antenna  must 
slave  to  boresight  in  all  axes  and  elevation  and  azimuth 
boresight  lockpins  must  engage  freely.  (5  thru  8) 


7.  Place  switches  and  controls  as  specified  in  table  4-1 
(as  shown  below) . 

Table  4-1  Initial  Control  Settings. 

Switch  or  Control  Position 

Intratarget  Data  Indicator 
Brilliance  Fully  Counterclockwise 
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Receiver  gain 
Contrast 


Fully  Counterclockwise 
Fully  Counterclockwise 
NORM 


Noma  1-Override 
Cursor  Control  Fully  Counterclockwise 

Nomal-hold-expand  NORM 

Autotilt  Deselected 

8.  Place  switches  and  controls  as  specified  in  table  4-1 
(as  shown  below) . 

Table  4-1  Initial  Control  Settings. 

Switch  or  Control  Position 


Radar  Set  Control 


Power 

Antenna  tilt 
Frequency 
Polarization 
Mode 


STBY 

0 (detent) 
4 

LIN 

AGR 


9.  Place  switches  and  controls  as  specified  in  table  4-1 
(as  shown  below) . 
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Table  4-1  Initial  Control  Settings. 

Switch  or  Control  Position 

Terrain  Clearance  &  Range  Set  Control 
Terrain  Clearance  2  X  100 

Range  Set  8 . 0 

Antenna  Scan  Power  Supply-Programmer 
Transmitter  disable  NORM 

Instrument  Panel 

Master  Function (switches)  Deselected (not  green) 

Heading  mode  MAN  HDG 

10.  Place  switches  and  controls  as  specified  in  table  4-1 
(as  shown  below) . 

Table  4-1  Initial  Control  Settings. 

Switch  or  Control  Position 

Tactical  Computer 

Present  Position  Toggle  UPDATE 

Computer  power  OFF 


Fault  locator 

Fault  locator  selector  OFF 

11.  Rotate  radar  altimeter  control  fully  clockwise  from  off 
(detent) . 

12.  Place  IMS  mode  select  switch  in  GRID  and  tactical 
computer  power  switch  in  PVfR. 

13.  Ensure  radar  set  power  switch  is  in  STBY  with  3-minute 
time  in  allowed.  Press  push-to-test  light  on  fault  locator. 
Safe  light  must  come  on.  (9) 


14.  Place  radar  set  control  switch  in  PHR,  select  GMP, 
select  lO-mile  range,  place  radar  set  control  switch  in 
STBY,  and  then  select  AGR. 
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15.  Pltice  fault  locator  selector  switch  in  BIT.  Fault 
locator  safe  light  must  come  on.  (10)  (Hit  backup  to  review 
note . ) 


NOTE 

Indicator  fail  light  will  come  on  in  all  fault 
locator  test  positions.  (This  note  applies  to  the 
last  step  on  the  screen.) 

16.  Place  fault  locator  selector  switch  in  ACFT.  Safe 
light  must  come  on.  (11)  Antenna  should  be  slaved  to  0 
degrees  in  pitch,  azimuth  and  roll.  (Hit  backup  to  review 
note . ) 


NOTE 

View  antenna  position  from  the  cockpit.  If  the 
antenna  is  stationary  assume  it  is  slaved  to  0 
degrees  in  pitch,  azimuth  and  roll.  (This  note 
applies  to  the  last  step  on  the  screen.) 

17.  Place  fault  locator  switch  in  P/S.  Safe  light  shall 
blink  once  and  must  stay  on.  (12)  Antenna  should  be 
scanning  in  CSTA.  (Hit  backup  to  review  note.) 


18.  View  the  antenna  from  the  cockpit.  If  the  antenna  is 
slewing,  assume  that  is  scanning  in  CSTA. 

19.  Place  frequency  selector  in  1. 

20.  Place  fault  locator  selector  switch  in  XNTR.  Safe 
light  must  come  on.  (13)  (Waveguide  switches  energize,  and 
antenna  should  be  scanning  in  CSTA.  Adjust  brilliance  and 
contrast  control  on  indicator  for  desired  display. 

Indicator  display  will  flash  on  and  off  at  the  elevation 
scan  rate.) 

21.  Place  frequency  control  in  frequency  positions  5, 3, 6, 2, 
in  that  order.  At  each  frequency  setting,  cycle  fault 
locator  switch  to  P/S  and  then  to  XMTR  and  observe  that  safe 
light  comes  on  in  each  position.  (13) 


22.  Place  fault  locator  selector  switch  in  CMPTR.  Safe 
light  must  come  on.  (15)  Antenna  should  be  scanning  in 
CSTA  and  indicator  must  be  blank. 

23.  Place  fault  locator  select  switch  in  A/R.  Verify  safe 
light  comes  on  within  10  seconds,  but  not  immediately.  (16 
&  17)  (Antenna  scans  in  CSTA  until  safe  light  comes  on  and 
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then  is  slaved  to  boresight.  Indicator  displays  a  flashing 
-t-/-  20  degrees  PPI  sweep  at  the  elevation  scan  rate  until 
antenna  goes  to  boresight;  then  indicator  is  blank. 

Waveguide  switches  are  energized  until  antenna  goes  to 
boresight,  and  then  switches  deenergize.) 

24.  Repeat  previous  step  for  frequency  selector  positions  4 
and  7.  Return  fault  locator  selector  switch  to  CMPTR  prior 
to  performing  previous  step  for  each  frequency  selected. 

Safe  light  must  come  on  in  each  frequency  selected.  (17) 
(Hit  backup  to  review  note) 


NOTE 

If  the  following  test  is  run  without  a  10-second 
wait  between  steps,  the  safe  light  will  return 
faster  each  time  it  is  run.  (Note  applies  to  step 
25) 

25.  Place  fault  locator  selector  switch  in  CMPTR  2.  Safe 
light  must  come  on.  Fault  locator  safe  light  may  be  on 
intermittently.  This  condition  is  acceptable.  (18) 

Antenna  should  be  scanning  in  CSTA  and  indicator  shall  be 
blank. 

26.  Place  fault  locator  selector  switch  in  CMPTR  3.  Safe 
light  must  come  on.  Antenna  should  be  scanning  in  CSTA  and 
indicator  should  blank.  Fault  locator  safe  light  may  be  on 
intermittently.  This  condition  is  acceptable.  (18) 

27.  Press  range  change/target  reject  switch  on  throttle 
quadrant  until  indicator  10  light  comes  on.  Place  fault 
locator  switch  in  SW  GEN.  Antenna  should  be  scanning  +/-  45 
degrees.  Safe  light  must  not  come  on. 

28.  Adjust  brilliance  and  contrast  control  on  indicator  to 

obtain  display  with  minimum  flicker.  Verify  display  is  128 
light  vertical  lines  separated  by  narrow  dark  lines  with  no 
discontinuities,  (fig*  4-2A) .  (19,  19A,  19B) 
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29.  Press  range  change/target  reject  switch  on  throttle 
quadrant  one  tine.  Observe  the  following: 

A.  Range  nust  change  from  10  to  5  miles.  (42) 

B.  Indicator  display  must  change  to  complement  of 
former  display  (light  lines  become  dark  and  narrow  dark 
lines  become  light).  (19  and  19A) 

30.  Press  range  change/target  reject  switch  on  throttle 
quadrant  one  time.  Verify  indicator  displays  -f/-  45  degrees 
PPI  scan  (shades  of  grey)  and  fault  locator  (fig.  4-2A)  safe 
light  comes  on.  (19) 

31.  Press  range  change/ target  reject  switch  on  throttle 
quadrant  one  time.  Verify  indicator  displays  memory  test 
No.  3  (fig.  4-2A) . 

NOTE 

Further  action  of  range  change/target  reject 
switch  cycles  radar  indicator  displays  between 
memory  test  No.  3  and  BITE  gray  shades  displays. 

(This  note  applies  to  the  last  step  on  the 
screen . ) 


32.  Place  frequency  selector  in  4  and  fault  locator 
selector  switch  in  SCAN  INTER.  (Hit  backup  to  review  note.) 

34.  The  end. 


WARNING 

THIS  IS  THE  LAST  SCREEN  OF  THE  TEST.  IF  YOU  SELECT 
BACKUP  YOU  CAN  REVIEW  THE  ABOVE  STEPS.  IF  YOU  ARE 
FINISHED  THEN  SELECT  NEXT. 
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Appendix  F;  Briefing  Instructions 


Introduction 

Thank  you  for  volunteering  to  be  a  test  subject  in  the 
evaluation  of  two  different  electronic  display  devices  for 
the  presentation  of  maintenance  technical  information. 

1  am  Captain  Jeff  Friend  and  this  is  Capt  Randy 
Gr instead.  He  are  graduate  students  at  the  Air  Force 
Institute  of  Technology  and  we  are  performing  this 
experiment  as  part  of  our  thesis  reguirement. 

P.VHP9^e 

The  purpose  of  this  experiment  is  to  compare  the 
performance  of  technicians  accomplishing  maintenance  tasks 
with  technical  data  presented  on  a  hand-held  display  system 
(Point  to  the  PNA  computer)  and  a  head-mounted  monocular 
display  system  (Point  to  the  HKDD) . 

This  experiment  is  sponsored  by  the  Armstrong 
Laboratory,  located  in  Area  B  at  Hright-Patterson  AFB,  as 
part  a  program  called  the  Integrated  Maintenance  Information 
System  or  INIS.  INIS  is  a  developmental  project  aimed  at 
providing  technicians  with  a  flight  line  computer  to  support 
maintenance  activities.  Some  of  these  activities  include 
providing  technical  data  for  specific  aircraft,  retrieving 
spare  parts  information  from  the  supply  computer,  providing 
automated  diagnostic  routines,  displaying  historical 
information  either  from  CAMS  or  the  wing,  and  supporting 
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training  requirements.  The  information  obtained  from  this 
experiment  will  support  the  selection  of  a  display 
technology  for  future  weapon  systems  such  as  the  F-22. 
Experimental  Description 

There  are  a  total  of  24  maintenance  technicians 
participating  in  this  experiment.  Technicians  have  been 
divided  into  two  groups,  full-time  guardsmen  and  traditional 
guardsmen.  Each  group  is  made  up  of  personnel  from  the 
Avionics  shop  and  the  APG  shop.  Each  technician  will 
receive  training  on  how  to  operate  the  two  display  devices. 
Then  each  person  will  perform  one  of  two  maintenance  tasks 
tailored  to  meet  the  requirements  of  the  experiment.  Those 
tasks  are  a  checkout  of  the  APQ-126  radar  and  an  inspection 
of  the  engine  bay  normally  performed  during  phase 
inspection.  Both  tasks  may  or  may  not  have  faults  or 
discrepancies  inserted  into  the  system  being  evaluated.  You 
will  be  asked  to  perform  a  task  using  the  portable  hand-held 
display  device  or  the  head-mounted  display  device.  Your  job 
will  be  to  perform  the  steps  displayed  and  identify  any 
faults  or  discrepancies  you  have  encountered.  We  will  be 
recording  the  amount  of  time  between  each  fault  detection 
and  total  task  completion  time.  You  are  encouraged  to  work 
as  quickly  as  possible.  After  you  complete  your  experiment, 
you  will  be  asked  to  complete  a  questionnaire  which 
addresses  certain  aspects  of  the  display  devices  and  the 
information  displayed.  We  also  will  be  performing  personal 
interviews  with  each  participant  to  hear  in  your  words  how 
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you  feel  about  the  various  display  devices.  The  information 
collected  in  this  experiment  will  not  be  associated  with 
your  name.  Participants  will  only  be  identified  by  subject 
number.  The  data  collected  will  not  be  related  to  your  job 
performance.  Your  supervisor  will  not  know  how  you  did  on 
the  experiment  nor  will  he  hear  any  of  the  comments  you 
provide  during  the  debriefing.  The  sequence  of  events  will 
be  as  follows: 

►  Random  assignment  of  subjects  to  experimental 
conditions. 

►  Introduction  to  the  experiment. 

►  Eye  examinations  will  be  performed. 

►  Written  operating  instructions  on  the  computer  to  be 
used  will  be  provided. 

►  Hands-on  computer  training. 

►  Technician  to  perform  experiment. 

►  Experimenters  to  debrief  technician,  administer  the 
questionnaire,  and  perform  a  personnel  interview. 
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Appendix  G;  Training  Instructions 


Using  the  Hand-held  Display 

You  will  be  using  a  specially  constructed  hand  held 
computer  for  displaying  the  steps  you  must  perform  during 

4 

this  experiment.  This  briefing  will  provide  you  with 
instructions  on  the  basic  operation  of  the  computer  before 
the  test  gets  underway.  Please  feel  free  to  ask  any 
questions  at  any  time  if  the  instructions  and  procedures  we 
describe  are  not  clear. 

The  information  contained  in  the  hand-held  computer  was 
extracted  from  engine  bay  checkout  workcards  and  the 
operational  checkout  of  the  APQ-126  radar  system.  Some 
modifications  have  been  made  to  the  procedures  to  facilitate 
this  experiment.  Three  workcards  from  the  engine  bay 
checkout  were  selected  and  tailored,  and  a  procedure  from 
T.O.  lA-70-2-8  was  inserted  into  the  checkout.  All  of  the 
steps  for  the  engine  bay  workout  will  be  performed  inside 
the  engine  bay  compartment  of  the  aircraft  and  the  total 
time  to  complete  the  task  should  be  around  thirty  minutes. 
The  fault  isolation  test  was  extracted  from  the  radar 
operational  checkout  maintenance  activity.  All  of  the  steps 
will  be  conducted  on  the  ground  immediately  beside  the 
aircraft  or  in  the  aircraft's  cockpit.  This  procedure 
should  take  approximately  thirty  minutes  as  well.  Although 
the  modified  procedures  have  been  greatly  reduced,  the 
wording  for  the  steps  selected  are  similar  to  what  you  would 
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normally  use  to  checkout  the  engine  bay  and  APQ-126  radar. 

The  screen  layout  is  a  straight  forward  configuration. 
At  the  top  of  each  screen  you  will  see  the  "title  bar", 
which  tells  you  what  task  you  are  performing.  At  the  bottom 
of  the  screen  is  the  "menu  line".  Shown  on  the  menu  line 
are  several  blocks  with  keywords  in  each.  For  the  purposes 
Of  this  experiment  these  blocks  can  be  ignored.  Directly 
above  the  menu  line  is  the  "instruction  line"  which  tells 
you  what  keystroke  to  perform  next.  In  the  remainder  of  the 
screen,  between  the  "title  bar"  and  "instruction  line",  is 
where  technical  information  will  be  displayed. 

When  appropriate,  NOTES,  CAUTIONS,  and  WARNINGS  will 
appear  on  the  screen  as  separate  text  boxes  overlaid  over 
the  steps  for  which  the  NOTE,  CAUTION,  or  WARNING  applies. 

On  the  lower  left  hand  corner  of  these  text  boxes  is  a 
rectangular  box  containing  the  word  OK  highlighted  by  means 
of  dark  double  lines.  The  double  lines  mean  this  box  is 
defaulted  to  the  NEXT  command.  In  this  experiment  the 
command  highlighted  will  always  be  the  NEXT  command.  After 
hitting  NEXT  the  NOTE,  CAUTION,  or  WARNING  will  disappear. 

If  at  any  time  you  wish  to  backup  and  review  a  NOTE, 

CAUTION,  WARNING  or  any  previous  screen,  the  keystroke  will 
be  3ACKUP.  One  final  point  about  the  NOTES,  CAUTIONS,  and 
WARNINGS.  On  each  you  might  see  a  scroll  bar.  All  of  the 
NOTES,  CAUTIONS,  and  WARNINGS  for  this  experiment  have  been 
segmented  so  that  all  of  the  text  is  visible  without  the  use 
of  the  scroll  bar,  ignore  it. 
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When  you  receive  the  computer,  it  will  be  already 
activated  and  displaying  the  first  steps  of  the  experimental 
task.  Once  we  instruct  you  to  begin,  you  will  go  through 
each  screen  performing  the  technical  instructions.  Do  only 
as  the  technical  instructions  state;  DO  NOT  perform 
troubleshooting  or  attempt  repair  as  you  normally  might. 

You  have  completed  the  performance  portion  of  the  experiment 
when  you  get  to  the  screen  containing  the  last  technical 
instructions.  To  remind  you  that  you  are  on  the  last  screen 
containing  technical  instructions  we  have  inserted  a 
CAUTION.  Read  the  CAUTION  and  then  hit  NEXT  to  complete  the 
last  screen  of  the  technical  instructions.  Because  we  are 
recording  completion  times  it  is  important  that  when  you 
complete  the  last  step  you  alert  one  of  us  that  you  have 
done  so.  Do  not  turn  off  the  computer.  Hand  the  computer 
to  one  of  us  at  the  completion  of  the  task.  Please  do  not 
discuss  any  of  the  discrepancies  you  may  have  found  during 
the  experiment.  There  will  be  11  other  members  from  your 
shop  performing  this  experiment  and  the  results  will  skewed 
if  they  know  in  advance  the  location  of  possible 
discrepancies . 

This  concludes  the  instructions  and  practice  on  the  use 
of  the  portable  laptop  computer.  Do  you  have  any  questions 
regarding  its  use  before  the  test  session  gets  underway? 

Using  the  Head-Mounted  Display  System 

You  will  be  using  the  head-mounted  display  system  as 
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the  source  of  checkout  maintenance  information  for  one  of 
the  tasks.  This  briefing  will  provide  you  with  instructions 
on  the  basic  operation  of  the  computer  before  the  test  gets 
underway.  Please  feel  free  to  ask  any  questions  at  any  time 
if  the  instructions  and  procedures  we  describe  are  not 
clear. 

The  information  contained  in  the  head-mounted  computer 
was  extracted  from  engine  bay  checkout  workcards  and  the 
operational  checkout  of  the  APQ-126  radar  system.  Some 
modifications  have  been  made  to  the  procedures  to  facilitate 
this  experiment.  Three  workcards  from  the  engine  bay 
checkout  were  selected  and  tailored,  and  a  procedure  from 
T.O.  1A-7D-2-8  was  inserted  into  the  checkout.  All  of  the 
steps  for  the  engine  bay  workout  will  be  performed  inside 
the  engine  bay  compartment  of  the  aircraft  and  the  total 
time  to  complete  the  task  should  be  around  thirty  minutes. 
The  fault  isolation  test  was  extracted  from  the  radar 
operational  checkout  maintenance  activity.  All  of  the  steps 
will  be  conducted  on  the  ground  immediately  beside  the 
aircraft  or  in  the  aircraft's  cockpit.  This  procedure 
should  take  approximately  thirty  minutes  as  well.  Although 
the  modified  procedures  have  been  greatly  reduced,  the 
wording  for  the  steps  selected  are  similar  to  what  you  would 
normally  use  to  checkout  the  engine  bay  and  APQ-126  radar. 

The  screen  layout  is  a  straight  forward  configuration. 
At  the  top  of  each  screen  you  will  see  the  "title  bar", 
which  tells  you  what  task  you  are  performing.  At  the  bottom 
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of  the  screen  is  the  "menu  line".  Shown  on  the  menu  line 
are  several  blocks  with  keywords  in  each.  For  the  purposes 
of  this  experiment  these  blocks  can  be  ignored.  Directly 
above  the  menu  line  is  the  "instruction  line"  which  tells 
you  what  keystroke  to  perform  next.  In  the  remainder  of  the 
screen,  between  the  "title  bar"  and  "instruction  line",  is 
where  technical  information  will  be  displayed. 

When  appropriate,  NOTES,  CAUTIONS,  and  WARNINGS  will 
appear  on  the  screen  as  separate  text  boxes  overlaid  over 
the  steps  for  which  the  NOTE,  CAUTION,  or  WARNING  applies. 

On  the  lower  left  hand  corner  of  these  text  boxes  is  a 
rectangular  box  containing  the  word  OK  highlighted  by  means 
of  dark  double  lines.  The  double  lines  mean  this  box  is 
defaulted  to  the  NEXT  command.  In  this  experiment  the 
command  highlighted  will  always  be  the  NEXT  command.  To 
execute  the  NEXT  command  depress  the  (red)  right  top  button 
on  the  pistol  grip.  After  hitting  NEXT  the  NOTE,  CAUTION, 
or  WARNING  will  disappear.  If  at  any  time  you  wish  to 
backup  and  review  a  NOTE,  CAUTION,  WARNING  or  any  previous 
screen,  invoke  the  BACKUP  command  by  depressing  the  (black) 
left  top  button  of  the  pistol  grip.  One  final  point  about 
the  NOTES,  CAUTIONS,  and  WARNINGS.  On  each  you  might  see  a 
scroll  bar.  All  of  the  NOTES,  CAUTIONS,  and  WARNINGS  for 
this  experiment  have  been  segmented  so  that  all  of  the  text 
is  visible  without  the  use  of  the  scroll  bar,  ignore  it. 

Using  the  HMDD  system  requires  the  user  make  some 
adjustments  prior  to  beginning  the  experiment.  The  head 
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band  size  is  adjusted  through  the  use  of  the  knob  on  the 
back  of  the  head  band.  The  location  of  the  eye  piece 
through  the  knob  on  the  side  of  the  headset.  To  adjust  the 
focus  on  the  display  device,  use  the  small  thumbwheel  on  the 
top  of  the  eye  piece.  We  will  help  you  with  the 
adjustments.  The  vest  should  also  be  fastened  and  adjusted 
so  that  the  wearer  is  comfortable.  When  you  receive  the 
computer,  it  will  be  already  activated  and  displaying  the 
first  steps  of  the  experimental  task.  Once  we  instruct  you 
to  begin,  you  will  go  through  each  screen  performing  the 
technical  instructions.  Do  only  as  the  technical 
instructions  state;  DO  NOT  perform  troubleshooting  or 
attempt  repair  as  you  normally  might.  You  have  completed 
the  performance  portion  of  the  experiment  when  you  get  to 
the  screen  containing  the  last  technical  instructions.  To 
remind  you  that  you  are  on  the  last  screen  containing 
technical  instructions  we  have  inserted  a  CAUTION.  Read  the 
CAUTION  and  then  hit  NEXT  to  complete  the  last  screen  of  the 
technical  instructions.  Because  we  are  recording  completion 
times  it  is  important  that  when  you  complete  the  last  step 
you  alert  one  of  us  that  you  have  done  so.  Do  not  turn  off 
the  computer.  We  will  help  you  with  its  removal.  Please  do 
not  discuss  any  of  the  discrepancies  you  may  have  found 
during  the  experiment.  There  will  be  11  other  members  from 
your  shop  performing  this  experiment  and  the  results  will 
skewed  if  they  know  in  advance  the  location  of  possible 
discrepancies . 


112 


This  concludes  the  instructions  and  practice  on  the  use 
of  the  HMDD  computer.  Do  you  have  any  questions  regarding 
its  use  before  the  test  session  gets  underway? 
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Appendix  Ht  Debriefing  Instructions 


We  are  grateful  for  your  participation  in  this 
evaluation.  The  purpose  of  the  experiment  was  to  compare 
the  performance  of  technicians  accomplishing  maintenance 
tasks  with  technical  data  presented  on  a  hand-held  display 
and  a  monocular  display  device.  The  F-22  and  B-2  are  both 
going  to  have  digital  technical  information  for  technicians 
on  the  flightline  and  in  the  shops.  The  information  from 
this  evaluation  will  support  the  selection  of  a  display 
technology  for  future  weapon  systems. 

The  information  received  from  user  evaluations  and 
personal  interviews  is  central  to  this  evaluation.  The 
results  of  this  study  will  be  available  to  all  federal 
agencies  who  wish  to  request  a  copy. 

None  of  the  information  received  or  data  collected  will 
be  associated  with  your  name.  Experimental  write-ups  will 
describe  the  data  only  by  subject  number.  Do  you  have  any 
other  comments  about  the  evaluation? 

We  will  return  to  the  wing  in  August  with  a  briefing  of 
the  experimental  results  after  data  analysis  has  been 
performed. 

Thanks  again  for  your  participation.  We  would 
appreciate  your  not  discussing  any  aspect  of  this  evaluation 
with  your  co-workers  until  all  of  the  data  has  been 
collected. 
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Appendix  I;  Observation  Form 


Subject  ID  _ 

1.  Task  _  HMDD  _  PMA  _ 

************************************************************ 

RADAR  CHECKOUT  COMMENTS/OBSERVATION: 

Start  Time: 

Fault  Find  Time: 

Termination  Time:  _ 

No.  of  Keystrokes:  N  B 
Correct:  Abort:  _ 


ENGINE  BAY  CHECKOUT 
Start  Time: 

Fault  Find  Time: 

Termination  Time:  _ 

No.  of  Keystrokes:  N _ B 

Correct :  _  Abort : 
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Appendix  J;  Task  Completion  Times  for  the  Overall  Sample 


Total  Completion  Times  for  Total  Sample 


1. 

HMDD/ EXP /TASK  1 

16.73 

15.  PMA/EXP/TASK  1 

22.34 

2. 

HMDD/ EXP/TASK  1 

31.85 

16.  PMA/EXP/TASK  1 

22.19 

3. 

HMDD/EXP/TASK  1 

22.05 

17.  PMA/EXP/TASK  1 

26.02 

4. 

HMDD/ EXP/TASK  1 

19.57 

18.  PMA/EXP/TASK  1 

30.26 

5. 

HMDD/EXP/TASK  2 

12.08 

19.  PMA/EXP/TASK  2 

11.61 

6. 

HMDD/EXP/TASK  2 

13.80 

20.  PMA/EXP/TASK  2 

16.01 

7. 

HMDD/EXP/TASK  2 

13.95 

21.  PMA/EXP/TASK  2 

11.80 

8. 

HMDD/INEXP/TASK 

1 

21.82 

22.  PMA/EXP/TASK  2 

17.89 

9. 

HMDD/INEXP/TASK 

1 

31.77 

23.  PMA/INEXP/TASK  1 

14.39 

10. 

HMDD/INEXP/TASK 

1 

15.78 

24.  PMA/INEXP/TASK  1 

22.89 

11. 

HMDD/INEXP/TASK 

1 

25.73 

25.  PMA/INEXP/TASK  1 

17.76 

12. 

HMDD/INEXP/TASK 

2 

13.97 

26.  PMA/INEXP/TASK  1 

23.94 

13. 

HMDD/INEXP/TASK 

2 

19.80 

27.  PMA/INEXP/TASK  2 

23.10 

14. 

HMDD/INEXP/TASK 

2 

17.02 

28.  PMA/INEXP/TASK  2 

29.  PMA/INEXP/TASK  2 

20.24 

28.62 

(Time  is  recorded  in  minutes  and  fractions  of  a  minute.) 


************************************************************ 


Missed  Detection  of  Inserted  Faults 


1  Engine  Bay  Checkout  Fault  Isolation  Test  | 

1  HMDD  PMA  1 

1  HMDD  PMA  1 

EXP 

1(4) 

i^BBI 

EXP 

mem 

mam 

_ UlL^ 

■H81 

INX 

1(3) 

1H8H 

(  )  Indicates  total  number  performing  checkout. 
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Appendix  K:  Factorial  Analysis  Results  for  Task  Completion 
Time  for  Radar  Fault  Isolation  Test 


Mean 

Exp 

Dis 

■SRS9 

BSBSWi 

1 

-1 

-1 

1 

23.99 

1 

1 

-1 

-1 

14.33 

1 

-1 

1 

-1 

16.93 

1 

1 

1 

1 

13.28 

4 

2 

2 

2 

17.13 

-6.66 

-4.06 

3.00  1  1 

Pooled  Estimate 

10.29 

Variance  of  Effect 

2.57 

Standard  Error 

1.60 

2*  Sigma 

3.21 
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pay  j.nspeci.j.on 


Mean 

Exp 

Dis 

Exp/Dls 

1 

-1 

-1 

1 

6.5 

1 

1 

-1 

-1 

5 

1 

-1 

1 

-1 

8.5 

1 

1 

1 

1 

13.5 

4 

2 

2 

2 

00 

n 

• 

00 

1.75 

5.25 

3.25  1  1 

Pooled  Estimate 

11.92 

Variance  of  Effect 

2.98 

Standard  Error 

1.73 

2*  Sigma 

3.45 
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Appendix  N;  Questionnaire  Breakdovm 


The  following  is  breakdo%m  of  each  subject's  responses 
to  each  question  shown  on  Appendix  B.  The  breakdown 
arrangement  follows  that  of  the  qpiestionnaire. 


HMDD 

A 

B 

C 

D 

Question  1 

2 

2 

7 

4 

Question  2 

0 

1 

8 

6 

Question  3 

1 

5 

6 

3 

Question  4 

2 

3 

6 

4 

Question  5 

0 

0 

4 

9 

Question  6 

2 

3 

7 

2 

Question  7 

4 

4 

4 

3 

Question  8 

6 

5 

1 

3 

Question  9 

5 

5 

5 

0 

Question  10 

5 

4 

5 

1 

Question  11 

4 

3 

7 

1 

Question  12 

3 

5 

6 

1 

PMA 

A 

B 

C 

D 

Question  1 

1 

6 

8 

0 

Question  2 

2 

6 

7 

0 

Question  3 

3 

7 

5 

0 

Question  4 

2 

9 

4 

0 

Question  5 

1 

11 

3 

0 

E 

0 

0 

0 

0 

2 

1 

0 

0 

0 

0 

0 

0 


E 

0 

0 

0 

0 

0 
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HMDD 

A 

B 

C 

D 

E 

Question  1 

2 

2 

8 

3 

0 

Question  2* 

1 

(2) 

5(5) 

2(2) 

0 

Question  3 

0 

1 

4 

9 

1 

Question  4 

1 

1 

4 

7 

2 

Question  5 

2 

3 

6 

2 

2 

Question  6 

3 

4 

8 

0 

0 

PMA 

A 

B 

C 

D 

E 

Question  1 

3 

4 

6 

2 

0 

Question  2 

1 

9 

5 

0 

0 

Question  3 

0 

3 

4 

5 

3 

IflIDD 

A 

B 

C 

D 

E 

Question  1 

3 

4 

6 

2 

0 

Question  2 

3 

4 

8 

0 

0 

PMA 

A 

B 

C 

D 

E 

Question  1 

1 

7 

7 

0 

0 

Question  2 

2 

9 

3 

1 

0 

*  (  )  Indicates  test  subjects  who  wore  the  keypad  input 
device  for  the  HMDD. 
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Appendix  O:  PMA  Interviews 


Background : 

5.  Radar,  inexp 
7.  Radar,  inexp 
10.  Engine  Bay,  inexp 

12.  Engine  Bay,  inexp 

13.  Engine  Bay,  inexp 

15.  Engine  Bay,  inexp 

16.  Radar,  inexp 

17.  Radar,  exp 

18.  Radar,  exp 

19.  Radar,  exp 

20.  Engine  Bay,  exp 

21.  Engine  Bay,  exp 

22.  Engine  Bay,  exp 

23.  Engine  Bay,  exp 

************************************************************ 
Question  1:  Is  the  information  on  the  screen  easy  to  read? 

5.  Yes. 

7.  Yes,  quite  clear. 

10.  Yes,  but  a  little  bit  of  it  was  breaking  up.  There  was 
a  line  that  interrupting  the  words  on  the  screen.  Other 
than  that  it  was  easy  to  read. 

12.  Yes,  It  was  real  easy  to  read 

13.  I  thought  it  was. 

15.  Some  of  it  was  a  little  difficult  to  read.  There  was 
some  horizontal  lines  that  would  disrupt  the  text. 

16.  No  problems.  It  was  real  easy  to  read. 

17.  When  there  was  a  lot  of  text,  it  would  be  nice  to  have 
more  spacing  between  tasks. 

18.  It  was  fine. 

19.  Under  the  test  conditions,  no  I  did  not  have  any 
problems.  I  would  be  concerned  under  adverse  flight  line 
conditions.  It  might  be  nice  to  have  a  stand  on  it  to 
adjust  the  tilt. 

20.  It  was  fine. 

21.  Very  easy  to  read. 
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22.  It  was  easy  to  read. 

23.  I  had  no  problems.  It  was  very  easy  to  read. 
************************************************************ 

Question  2:  Give  some  examples  of  other  activities  that 
would  be  enhanced  by  using  this  device. 

5.  Yes,  When  you  do  a  IMS  drifting  procedure.  It  is  a  step 
by  step  process.  This  device  would  be  very  helpful.  Also 
when  you  are  conducting  a  Maverick  missile  check.  There  are 
allot  of  switches  that  must  be  set  in  the  correct  order.  If 
you  get  one  switch  off  you  can  throw  the  whole  thing  off. 

7.  Trouble  shooting  activities  that  require  you  look  up 
information  in  several  different  T.O.s.  Using  the  branching 
function  to  go  directly  to  the  procedure.  Saves  downtime. 

10.  No.  I  can't  think  of  any.  Actually  It  would  have  been 
easier  to  perform  the  engine  bay  inspection  with  a 
checklist.  In  the  engine  bay,  there  was  no  place  to  set  the 
computer  down  safely.  I  had  to  jump  in  and  out  to  see  what 
to  do  next.  You  could  try  to  wear  it  over  your  shoulder 
,but  the  way  the  strap  is  designed,  you  would  have  to  read 
the  screen  upside  down.  Plus,  you  would  be  more  likely  to 
knock  the  screen  against  a  fitting  or  something  and  break 
the  screen. 

12.  Just  about  any  job  would  be  suitable.  Especially  if  the 
job  requires  schematics  or  wiring  diagrams. 

13.  You  could  use  computer  on  anything  that  you  do  on  the 
aircraft.  You  can't  hold  it  all  of  the  time,  there  are  some 
operations  that  you  can't  hold  a  book  the  whole  time. 

15.  I  am  still  partial  to  the  workcards.  You  are  not  afraid 
to  get  them  greasy.  You  can  wipe  them  off  and  stick  them  in 
your  pocket.  They  are  more  portable. 

16.  For  tasks  that  are  in  the  cockpit  the  device  would  work 
real  well.  Activities  that  don't  require  you  to  move 
around  allot  to  different  areas  of  the  aircraft.  For 
example,  drift  checks. 

17.  I  can't  think  of  anything. 

18.  There  could  be  allot  of  applications.  The  IMS  alignment 
and  drift  test  procedure  would  be  a  good  one  to  use.  The 
Inertial  Measurement  System  alignment  requires  check  certain 
points  with  the  IMS  at  different  settings.  You  then  drift 
IMS  system  and  watch  for  it  to  feel  velocities  or  movement. 
We  are  required  to  record  data  every  five  minutes  throughout 
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the  drift  cycle.  The  whole  procedure  takes  about  an  hour. 
The  rest  of  the  operational  checkout  of  the  radar  would  also 
be  a  good  test. 

19.  I  thin':  it  is  unlimited  as  to  what  tasks  it  could  be 
used  for  on  the  airplane. 

20.  The  engine  bay  is  a  good  example  because  you  have  to 
refer  to  several  T.O.s.  In  most  of  my  work,  I  use  work 
cards.  The  work  cards  have  only  the  minimiun  that  you  have 
to  inspect  during  a  phase^l  inspection.  That  should  not  be 
"THE”  inspection.  Everything  in  the  engine  bay  is  subject 
to  my  inspection.  The  work  cards  are  handy  to  have  to  make 
sure  that  you  have  checked  specific  items  of  interest. 

21.  Mostly  with  specialist.  Guys  who  work  in  the  cockpit. 
When  they  need  to  out  in  the  weather  and  windy  conditions. 

I  can  see  this  device  being  more  useful  on  aircraft  that  are 
more  complex,  the  F-16  and  F-18  have  allot  more  electronic 
equipment  onboard.  The  maintenance  tasks  involved  in  those 
aircraft  would  require  a  larger  amount  of  data  in  different 
T.O.  volumes.  Having  all  the  data  accessible  from  one 
screen  would  be  a  benefit. 

22.  None  that  I  can  think  of. 

23.  Those  tasks  that  require  more  than  one  T.O.  would 
benefit  from  a  device  like  this. 

************************************************************ 

Question  3:  What  did  you  not  like  about  the  device. 

5.  Going  from  screen  to  screen  was  too  slow. 

7.  Slow  response  time,  easy  to  operate 

10.  It's  size  and  its  weight.  It  was  nice  that  it  had  a  big 
screen,  but  overall  I  thought  it  would  be  cumbersome  to  use. 


12.  Took  a  long  time  between  screens.  The  size  and  weight 
may  cause  some  people  to  avoid  using  it. 

13.  No  it  was  fine.  It  was  easy  to  operate. 

15.  Lack  of  durability.  You  don't  want  to  damage  it.  It 
isn't  as  portable  as  workcards. 

16.  It  worked  real  well. 

17.  The  screen  tended  to  lock  up  when  I  would  press  backup. 
The  long  processing  time  contributed  to  the  problem. 
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18.  No. 


19.  The  slow  response  time  and  fact  that  there  appears  to  be 
a  heat  problem  that  causes  the  system  to  halt.  Also  the 
problem  with  the  system  halting  if  the  operator  presses  the 
next  or  backup  command  too  many  times  in  a  row. 

20.  The  size  and  weight  are  a  disadvantage.  Right  now  I  can 
use  work  cards,  that  I  don't  have  to  worry  about  damaging, 
while  I  am  performing  the  maintenance  action. 

21.  In  the  case  of  an  engine  bay  inspection,  I  cidn't  see  a 
whole  lot  of  advantage  of  using  the  computer.  I ^  is  a  good 
training  device.  Because  I  am  so  familiar  with  the 
workcards,  I  looked  at  the  data  on  the  screen  and  performed 
my  checks.  I  didn't  have  to  go  back  to  the  computer  except 
for  advancing  to  the  next  screen.  Nhen  I  do  an  engine  bay 
inspection  I  look  at  everything  in  there.  I  don't  just  look 
at  what  is  on  the  work  cards. 

22.  It  would  have  been  nice  to  have  something  you  could  take 
into  the  engine  bay  with  you.  I  like  to  have  something  with 
me.  I  didn't  like  having  to  crawl  out  each  time  to  look  at 
the  T.O.  I  would  be  afraid  of  damaging  the  system.  I  needs 
to  be  made  more  rugged.  I  would  be  less  likely  to  damage  a 
paper  T.O.  When  I  was  doing  the  inspection,  I  would  read 
the  three  tasks  on  the  screen  .and  then  perform  them.  It 
would  have  been  nice  to  refer  back  to  the  tasks  without 
having  to  go  back  out  to  the  computer. 

23.  No.  It  worked  fine. 

************************************************************ 

Question  4:  Were  there  any  Visual  problems  with  reading  the 
screen  (problems  with  glare)? 

5.  No.  There  was  no  glare. 

7.  No  the  operation  was  in  the  hanger,  I  had  no  problem  with 
seeing  the  screen. 

10.  No.  It  was  pretty  good.  I  could  read  the  screen  from  any 
angle. 

12.  No  problem  with  glare.  I  think  it  was  easier  to  read 
than  if  it  were  in  print  form.  Plus  you  can  read  it  at 
night. 

13.  No.  It  was  cloudy  out.  I  didn't  have  any  problems 
reading  it.  I  could  read  the  text  standing  up.  I  tended  to 
read  it  closer  up  so  I  wouldn't  miss  anything. 
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15.  No.  It  was  pretty  readable. 

16.  There  was  some  glare.  I  used  my  hand  to  get  rid  of  the 
shadows  so  I  could  be  sure  to  read  the  instructions 
correctly. 

17.  The  position  of  the  device  created  a  situation  in  which 
the  hanger's  overhead  lighting  was  causing  glare  on  the 
screen. 

18.  No  problems. 

19.  No  I  had  no  problems  with  glare. 

20.  It  was  fine. 

21.  No.  I  had  no  problems  with  glare. 

22.  No,  it  was  legible,  there  was  one  part  where  it  seemed 
bunched  up.  The  words  could  be  spaced  out  more. 

23.  No.  If  glare  was  a  problem,  it  wouldn't  be  difficult  to 
construct  a  hood  to  shield  the  screen.  I  didn't  have  any 
problem  with  it. 

Question  5:  Was  the  task  you  performed  representative  of 
maintenance  activities  you  are  asked  to  routinely 
accomplish? 

5.  Yes. 

7.  Yes. 

10.  Sure.  It  was  a  fair  representation. 

12.  Yes  fairly  typical. 

13.  Usually  the  technicians  perform  the  engine  bay 
inspection. 

15.  Not  so  much  the  engine  bay.  Its  not  that  often  we  pull  a 
engine.  For  the  regular  guys  it  would  be. 

16.  Yes  we  will  do  a  radar  checkout  almost  daily. 

17.  Yes  it  is  one  of  the  more  common  things  we  do. 

18.  Yes  it  may  be  to  representative.  I  am  very  familiar 
with  this  procedure  and  tend  to  deviate  from  the  T.O. 
instructions.  When  I  do  a  radar  checkout,  I  am  looking  for 
the  rather  obvious.  When  I  get  the  debrief,  I  usually  have 
a  good  idea  as  to  where  the  problem  is.  I  won't  normally 
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perform  the  radar  checkout  as  it  is  written  in  the  T.O. 

19.  The  task  was  pretty  basic.  Allot  of  the  system 
checkouts  would  be  more  difficult  than  the  fault  isolation 
test. 

20.  Yes.  If  it  weren't  timed  I  would  have  taken  half  hour 
more.  Usually  you  inspect  the  engine  bay  with  the 
specialist  there  from  the  different  shops.  So  as  you  are 
going  through  it  with  them  you  are  inspecting  the  whole  time 
they  are  inspecting.  Then  you  go  in  and  perform  your 
checks . 

21.  Similar,  yes.  The  type  of  inspection  would  be  the  same. 

I  would  normally  inspect  more  of  the  systems  in  the  engine 
bay. 

22.  Yes.  It  is  my  normal  job. 

23.  Yes.  The  engine  bay  is  one  of  those  places  where  it  is 
difficult  to  carry  any  T.O.  material  in  with  you  regardless 
if  it  is  a  book  or  a  computer. 

************************************************************ 

Question  6:  Looking  at  the  manner  in  which  we  conducted  the 
experiment,  including  the  prebrief,  instructions,  eye 
examination,  experiment,  survey,  and  interview,  would  you 
like  to  see  any  changes  made?  Was  the  information  adequate? 


5.  No. 


7.  No  changes.  Instructions  were  clear. 

10.  I  thought  all  the  instructions  were  clear  and 
straightforward.  It  was  easy  to  operate  the  computer. 

12.  It  was  satisfactory.  It  was  real  simple  to  use. 

15.  No.  the  instructions  were  satisfactory. 

16.  You  might  consider  using  schematics.  Schematics  would 
be  useful  for  chasing  down  wires  during  troubleshooting.  It 
would  be  easier  than  the  fold  out  charts  the  T.O.s  have.  We 
don't  chase  down  wires  everyday  but  when  we  do  this  device 
would  make  it  easier. 

17.  It  would  have  been  nice  to  have  a  dry  run  to  get  used  to 
using  the  device  and  the  test  environment. 


18.  No. 


19.  No. 
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20.  No. 


21.  I  would  you  try  a  wider  application.  I  would  test  more 
of  the  computer's  capabilities. 

22.  Your  eye  test  is  for  long  distance.  I  think  that  it 
would  be  better  to  test  vision  close  up. 

23.  The  test  article  could  have  had  higher  fidelity. 

Because  this  aircraft  is  a  hanger  queen.  It  tends  to  have 
parts  cannibalized  off  of  it.  So  when  you  are  doing  the 
inspection,  you  are  not  certain  if  things  missing  from  the 
aircraft  is  considered  a  discrepancy  or  is  the  result  of  the 
aircraft  not  being  operational.  An  operational  aircraft  I 
would  be  able  to  spot  the  discrepancy  right  off. 

****************A**********ik****************4t*******4t4r****** 

Question  7:  Do  you  have  any  other  questions  or  comments? 


5.  No. 


7.  No. 


10.  No. 


12.  No. 

13.  The  concept  is  a  good  one.  You  might  see  if  there  could 
be  a  radio  link  to  a  base  computer  for  checking  the 
maintenance  history  of  that  airplane.  The  unit  seems  small 
enough  that  it  will  fit  in  the  cockpit  or  you  could  hang  it 
by  the  strap. 

15.  You  should  look  at  making  the  system  smaller.  If  it  was 
palm  size,  then  it  could  fit  in  your  pocket.  More  work  card 
size. 


16.  No. 


17.  No. 

18.  I  thought  that  the  crystal  screen  was  broken  up  at 
times.  There  are  lines  that  interrupted  the  text.  The  text 
was  still  readable.  It  would  be  more  of  a  problem  with  a 
schematic. 

19.  Just  that  when  the  bugs  get  all  worked  out,  it  will  be  a 
good  system. 

20.  I  can  see  an  advantage  to  having  the  computer  would  be 
the  ability  to  update  T.O.  much  more  quickly.  Sometimes  we 
are  a  week  behind  in  placing  all  of  the  updates  into  the 
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T.O.s.  By  using  a  computer  I  could  make  the  updates  much 
more  quickly. 

21.  I  would  think  the  HMDD  would  be  more  of  a  hinderance 
than  the  hand  held  computer.  You  can  set  aside  the  hand  held 
when  you  are  not  using  it. 

22.  No. 

23.  No. 
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Appendix  P;  HMDD  Interviews 


Background 

1.  Radar, 

2 .  Radar , 

3.  Radar, 

4.  Radar, 

8 .  Engine 

9.  Engine 
11.  Engine 
14 .  Engine 

24.  Radar, 

25.  Radar, 

26.  Radar, 

27.  Radar, 

28.  Engine 

29.  Engine 

30.  Engine 

31.  Engine 


dominant  eye ,  inexp 
dominant,  inexp 
dominant ,  inexp 
non-dominant,  inexp 
bay,  non-dominant,  inexp 
bay,  non-dominant,  inexp 
bay,  dominant  eye,  inexp 
bay,  dominant  eye,  inexp 
dominant  eve,  exp 
dominant  eye ,  exp 
dominant  eye,  exp 
dominant  eye ,  exp 
Bay,  dominant  eye,  exp 
Bay,  dominant  eye,  exp 
bay,  non-dominant  eye,  exp 
Bay,  dominant  eye,  exp 


************************************************************ 


Question  1:  Is  the  information  on  the  screen  easy  to  read? 

1.  Yes,  it  was. 

2.  Yes  and  no.  It  was  easy  to  read  as  long  as  I  had  a  clear 
image  of  the  screen.  I  had  to  keep  adjusting  it.  It  would 
have  been  easier  if  there  was  a  larger  boarder  around  the 
information.  The  edges  were  the  toughest  to  read. 

3.  Yes.  I  had  problems  focusing  on  the  full  range  of 
information.  I  wear  no  line  bifocals  and  I  was  unable  to  see 
the  whole  screen  clearly.  If  I  moved  it  around,  I  could  see 
read  all  of  the  text. 

4.  It  was  when  the  screen  was  adjusted  correctly. 

8.  The  text  was  O.K.  but  at  times  it  seemed  the  text  run 
together  however.  The  letter  W  and  letter  H  were  hard  to 
read. 

9.  Yes. 

11.  Once  I  used  to  focusing  on  the  screen  yes.  Placing  the 
screen  closer  to  eye  made  it  easier  to  see  the  whole  screen. 
At  first,  I  could  not  see  the  top  left  hand  part  of  the 
screen. 

14.  Yes 

24.  It  was  excellent. 

25.  It  was  fairly  easy  to  read. 
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26.  I  had  a  hard  time  focusing.  I  wear  reading  glasses  and 
it  was  difficult  for  me  to  read  the  screen  with  it  being  so 
close  to  my  eye. 

27.  I  found  the  information  easy  to  read. 

28.  It  was  fine. 

29.  I  had  problems  get  it  adjusted  so  I  could  read  the  text. 

30.  It  wasn't  too  bad.  I  could  read  it. 

31.  It  was  little  bit.  It  was  a  little  too  close  for  me.  I 
am  far  sighted  so  I  had  a  hard  time  focusing. 

************************************************************ 

Question  2:  Give  some  examples  of  other  activities  that 
would  benefit  from  using  this  device. 

1.  Any  time  you  are  out  in  the  weather  this  device  would  be 
preferred  over  a  paper  T.O.  It  would  be  especially  helpful 
for  the  guardsmen.  We  come  out  once  a  month  and  it  is  nice 
to  have  a  reference  tool.  T.O.  are  difficult  to  use  if  you 
are  not  familiar  with  them. 

2.  Anything  where  you  would  need  to  be  balancing  yourself. 
Any  situation  where  you  need  to  keep  you  hands  free. 
Specifically  any  radar  checkouts  where  you  need  the 
information  right  there  in  front  of  you. 

3.  Checking  the  PAVE  PENNY  (laser  guided  system)  general 
radar  checks,  any  work  with  the  onboard  computer. 

4.  Performing  a  Maverick  check  maybe  a  possible  activity.  It 
requires  somebody  in  the  cockpit  and  somebody  on  the  ground 
at  the  same  time.  The  HMDD  would  make  it  possible  for  both 
people  to  see  what  is  going  on.  It  would  be  better  than 
yelling  back  and  forth  because  you  have  to  wear  a  head  set 
allot  of  times. 

8.  It  might  be  useful  on  a  combat  turnaround  because  we  are 
working  real  fast  and  we  don't  have  time  to  pull  out  a  chart 
to  see  what  to  do  next. 

9.  I  would  limit  it  to  inspections  only. 

11.  For  my  activities  it  would  be  a  hinderance.  It  does  not 
provide  any  real  advantage  of  something  you  could  set  on  the 
ground . 

14.  Pretty  much  anytime  we  use  a  T.O.  this  device  would  give 
us  quick  access  to  the  information.  Better  than  looking  back 
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and  forth  through  the  pages.  A  lap  top  you  have  to  worry 
about  dropping  it,  Using  this  device  you  would  be  less 
likely  to  damage  it.  Most  of  the  time  you  want  to  keep  your 
hands  free  to  do  the  work.  By  not  having  the  device  on  you, 
the  laptop  computer  is  another  piece  of  equipment  you  have 
to  worry  about  safeguarding. 

24.  I  can't  think  of  any.  I  am  set  in  my  ways.  I  have  been 
doing  my  job  for  16  years  with  a  book  in  front  of  me. 

25.  When  you  are  out  looking  in  the  radome  itself,  it  would 
be  handy  to  have  the  HMDD.  Anytime  you  are  working  outside 
the  aircraft  when  it  is  windy. 

26.  This  device  would  be  useful  for  anytime  you  have  a  T.O. 
page  that  involves  foldout  pages.  You  could  read  the  tech 
data  and  view  the  hardware  at  the  same  time. 

27.  I  would  be  great  for  special  activities.  I  wouldn't  use 
it  for  routine  operations  that  you  perform  all  the  time.  I 
would  use  it  more  for  troubleshooting  complex  equipment. 

28.  I  can't  think  of  any.  It  would  be  easier  to  use  a  book. 

29.  For  my  self,  I  thought  it  hindered  me.  I  felt  like  I 
had  only  the  use  of  one  eye  for  inspecting  the  hardware. 

30.  Engine  bay  is  a  good  example.  Any  place  where  you  would 
do  inspections. 

31.  Any  task  where  you  need  to  move  around  allot. 
************************************************************ 

Question  3:  What  did  you  not  like  about  the  device. 

1.  It  was  cumbersome.  There  seemed  to  be  allot  of  bulk 
especially  the  headset.  It  wasn't  anything  major. 

2.  If  I  was  out  in  95  degree  weather,  that  vest  would  have 
been  unbearable.  The  equipment  on  the  back  is  uncomfortable 
when  sitting. 

3.  I  felt  it  was  a  little  bulky.  I  understand  it  is  a 
prototype . 

4.  I  had  some  problems  adjusting  the  eyepiece  there  in  the 
beginning.  It  took  awhile  to  get  it  just  right.  The  pack 
was  getting  hot  there  in  the  end. 

8.  It  seemed  that  the  locking  arm  was  a  little  cumbersome. 
Also  the  tilting  of  the  text  inside  the  screen  has  hard  to 
adjust  it  just  right. 
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9.  I  would  be  concerned  with  extended  wear  of  the  device  as 
well  as  wear  during  adverse  weather  conditions.  If  it  is 
100  degrees  outside,  I  couldn't  imagine  wearing  the  headset 
for  a  few  minutes  let  alone  over  an  hovir. 

11.  It  took  awhile  to  position  the  screen.  The  device  seems 
to  be  in  your  way.  It  blocks  your  eye. 

14.  It  took  awhile  to  get  used  to.  If  I  could  put  it  on  my 
dominant  eye,  it  may  have  been  easier  to  focus.  When  I  was 
inspecting  a  dark  area  I  would  have  a  hard  time  focusing 
past  the  eyepiece. 

24.  I  disliked  the  bulk  of  the  headset  and  the  glare  on  the 
screen.  I  would  be  concerned  on  how  much  abuse  it  could 
take. 

25.  Using  the  system  inside  the  cockpit  is  a  bit 
uncomfortable.  The  battery  pack  and  stuff  on  the  back  got 
in  the  way. 

26.  I  had  trouble  seeing  the  instruments  in  the  cockpit. 

27.  I  disliked  the  display  blocking  my  vision.  I  had 
difficulty  in  viewing  the  fault  location  switch  with  the 
device  on.  It  may  be  that  I  wasn't  used  to  wearing  it. 

28.  I  disliked  the  weight  of  the  headset  and  having  the 
eyepiece  right  in  front  of  my  field  of  view. 

29.  I  thought  it  blocked  my  vision.  If  you  had  to  wear  it 
for  extended  period  time,  it  would  become  hot. 

30.  I  noticed  it  has  a  real  high  frequency  flicker.  It  was 
annoying  after  awhile. 

31.  The  actual  screen  text  looked  crooked  in  the  lens.  That 
was  distracting. 


************************************************************ 

Question  4:  Were  there  any  Visual  problems  with  reading  the 
screen  (problems  with  glare)? 


1.  No. 

2.  I  had  problems  reading  the  text  in  the  very  top  and 
bottom  of  the  screen.  I  had  no  problems  changing  the  depth 
of  field  when  viewing  the  text  and  the  cockpit  instrtiments . 


3.  No.  I  did  notice  that  I  would  close  my  other  eye  when 
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viewing  the  screen. 

4.  At  first  I  had  to  squint  because  I  wasn't  used  to 
distinguishing  the  background  from  the  image  displayed  in 
front.  After  awhile,  I  was  able  to  view  the  screen  through 
the  background. 

8.  No. 

9.  When  I  looked  out  from  the  engine  bay  to  the  outside,  the 
bright  sunlight  completely  wiped  out  the  screen.  I  could  see 
it  fine  as  long  as  I  was  facing  the  engine  bay,  away  from 
the  sunlight. 

11.  I  had  no  problem  with  glare.  I  had  difficulty  in 
focusing  on  objects. 

14.  I  did  until  I  moved  it  closer  to  my  eye.  I  was  getting 
reflection  off  my  glasses. 

24.  I  had  problems  with  the  glare.  I  had  to  position  my  head 
so  I  could  read  the  screen  without  the  light  bouncing  off 
the  screen. 

25.  It  took  a  little  while  to  get  used  to  the  eye  piece. 
After  the  first  couple  screens,  it  was  easier  to  focus.  I 
didn’t  have  any  problem  with  glare. 

26.  When  I  was  trying  to  adjust  an  instrument  in  the 
cockpit,  it  was  difficult  to  see  it  because  of  the  light  in 
my  dominant  eye.  Maybe  a  switch  to  turn  the  screen  off 
would  be  useful. 

27.  I  found  that  I  was  closing  one  eye  to  read  the  screen.  I 
had  difficulty  in  reading  the  screen  with  both  eyes  open. 


28.  No. 

29.  There  seemed  to  be  allot  of  background  glare. 


30.  No. 


31.  No. 

************************************************************ 

Question  5:  Did  you  suffer  any  headaches  or  eye  strain  from 
using  the  device? 

1.  No.  I  am  feeling  a  little  stress  from  using  the  HMDD.  It 
is  probably  from  switching  from  looking  close  and  then  far. 

2.  Yes  while  trying  to  read  the  text  in  the  marginal  areas  I 
was  experiencing  some  eye  strain. 
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3.  No. 


4.  No. 


8.  No. 

9.  Just  a  little.  Things  appear  brighter  through  my  right 
eye(  eyepiece  was  on  the  right  eye)  than  my  left.  My  right 
pupil  might  slightly  dilated. 


11.  No. 


14.  No. 

24.  If  it  had  gone  on  for  a  longer  period  of  time,  I  might 
have.  I  am  a  little  tense  now. 

25.  No 


26.  No. 


27.  No. 


28.  A  little  bit  of  eye  strain.  No  headache  though. 

29.  No,  not  really.  If  I  had  to  wear  it  longer,  I  think  I 
would  have  some  eye  strain. 

30.  No. 


31.  No. 


************************************************************ 

Question  6:  Did  you  have  any  problems  with  the  headband? 


1.  No. 

2.  It  is  a  little  bit  on  the  heavy  side  and  it  makes  you 
sweat. 

3.  I  could  feel  the  weight  but  it  wasn't  a  problem. 

4.  It  was  O.K.  It  didn't  seem  too  heavy. 

8.  Though  I  had  the  headband  on  tight,  it  seemed  that  the 
headset  would  start  to  fall  off  when  I  looked  up.  It  seemed 
top  heavy. 

9.  It  is  fine  when  you  are  not  wearing  anything  else  on  your 
head.  It  might  be  hard  to  wear  with  ear  protectors  and  or 
goggles. 
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11.  The  headband  seemed  a  little  heavy.  It  would  slip  every 
once  and  a  while. 

14.  When  I  got  it  tight  enough  that  it  wouldn't  move,  it 
became  uncomfortable  to  wear  after  awhile.  I  needed  it 
steady  since  I  was  climbing  around.  I  did  not  have  any 
problems  with  it  moving  around.  It  would  have  been  nice  to 
be  able  to  move  eyepiece  around  incrementally  instead  of 
locking  it  in  place. 

24.  No  problem.  It  fit  O.K. 

25.  Yes,  it  was  too  big.  I  couldn't  adjust  it  do%m  enough  to 
fit.  I  could  wear  it  with  a  baseball  cap  however. 

26.  No. 

27.  I  disliked  the  whole  mechanism.  It  seemed  very  bulky. 

28.  The  fact  that  it  weighed  too  much  and  you  had  to  have 
something  on  your  head. 

29.  There  was  some  problems  with  getting  the  adjustments 
just  right  so  I  could  read  the  screen. 

30.  It  moved  around  a  little  bit.  I  banged  it  a  couple 
times.  I  don't  think  I  had  it  tight  enough.  The  eyepiece 
stayed  in  place  fairly  well. 

31.  It  tended  to  slip  a  bit  when  I  tilted  my  head  back.  I 
had  to  adjust  several  times  while  I  was  working. 

************************************************************ 

Question  7:  Did  you  have  any  problem  with  the  vest? 

1.  No,  not  really.  It  was  a  little  cumbersome  but  not 
restrictive. 

2.  No.  It  didn't  get  in  the  way. 


3.  No. 


4.  No.  It  seemed  all  right.  The  weight  wasn't  a  problem,  it 
seemed  to  weigh  about  the  same  as  a  bulletproof  vest  would. 

8.  It  seemed  O.K.  I  didn't  like  having  the  keypad  on  my  arm. 
I  would  be  afraid  of  rubbing  it  up  against  something  without 
knowing. 

9.  I  was  surprised  that  it  didn't  take  any  time  to  adapt  to 
wearing  the  vest.  It  fit  comfortably  and  wasn't  as  heavy  as 
it  looked.  It  wasn't  as  cumbersome  as  I  had  perceived  it 
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would  be.  I  wouldn't  want  to  have  to  do  maintenance 
activities  with  the  vest  on  because  the  vest  would  reduce  my 
access  into  small  compartments.  Also  I  would  be  concerned 
with  leaving  FOD  in  a  critical  part  of  the  aircraft.  The 
vest  adds  to  amount  of  things  that  can  get  hung  up  or  drop 
off  into  aircraft  compartments. 

11.  It  seemed  to  get  warm  on  the  lower  portion  of  the  back. 
The  weight  was  distributed  fairly  evenly.  It  was 
comfortable  to  wear. 

34.  The  vest  was  warm.  It  wasn't  a  function  of  the 
electronic's  generating  heat  but  more  a  function  of  me 
moving  around  with  it  on.  You  may  consider  making  the  vest 
out  of  a  mesh. 

24.  I  liked  the  key  pad  on  the  biceps.  That  was  handy  to 
use. 


25.  It  seemed  fine.  When  you  sat  in  the  cockpit,  you  knew 
you  had  the  battery  packs  were  there.  It  wasn't  unbearable, 
just  a  hinderance. 


26.  No. 

27.  It  was  all  right.  I  just  don't  like  having  extra  things 
on  while  I  am  working. 

28.  I  preferred  to  have  the  keypad  on  the  vest  instead  of  on 
my  bicep. 

29.  It  seems  bulky  and  the  batteries  don't  last  long  enough. 
Once  it  was  adjusted  it  stayed  in  place  fairly  well. 

30.  I  used  the  pistol  grip  input  device.  I  would  prefer  the 
keypad  on  the  arm  band. 

31.  It  was  a  little  hot  in  this  weather.  It  seem 
distributed  pretty  well. 

************************************************************ 

Question  8:  Was  the  task  you  performed  representative  of 
maintenance  activities  you  are  asked  to  routinely 
accomplish? 

1.  Usually,  you  would  have  a  partner  outside  the  cockpit  to 
help  examine  the  airplane  and  radome. 

2.  Yes. 

3.  Yes. 
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4.  Yes. 


8.  Right  now  we  don't  do  allot  of  engine  inspections,  we 
don't  do  allot  of.  But  when  we  get  the  F-16  we  will  be  doing 
more  of  them. 

9.  No.  It  is  a  specialty  function.  We  perform  phased 
inspections  periodically.  It  is  not  a  daily  activity.  Most 
of  the  work  is  on  schedule  and  you  have  to  work  as  fast  as 
you  can  get  the  planes  turned  around.  An  engine  bay 
inspection  is  not  a  time  critical  maintenance  acrion.  There 
is  no  real  time  limit. 

11.  I  have  never  performed  an  engine  bay  inspection.  This 
was  the  first  time  I  have  been  in  an  engine  bay.  If  I  was 
full  time,  I  might  have  more  experience  with  the  task. 

14.  Yes. 

24.  Yes>  very  much  so. 

25.  Yes,  anytime  we  have  a  write-up  on  the  radar  we  perform 
the  fault  isolation  test  first. 

26.  Yes. 

27.  Yes. 

28.  Yes. 

29.  Yes. 

30.  Yes,  it  seemed  to  be  realistic.  If  you  were  worried 
about  bumping  it  around  or  snagging  it  on  something,  the 
engine  bay  is  a  good  place  to  test  it. 

31.  Yes. 

Question  9:  Looking  at  how  we  conducted  the  experiment, 
including  the  prebrief,  instructions,  eye  examination, 
experiment,  survey,  and  interview,  would  you  like  to  see  any 
changes  made?  Was  the  information  adequate? 

1.  It  seemed  like  some  instructions  were  a  little  vague.  I 
was  not  familiar  enough  with  all  the  controls  to  be  sure 
what  the  instructions  were  asking  me  to  do. 

2.  No,  it  seemed  to  go  fairly  well. 

3.  It  would  have  been  nice  to  have  pictorials.  As  guardsmen, 
we  are  not  that  familiar  with  the  systems  being  tested. 
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4.  No,  I  think  you  made  everything  pretty  clear.  I  didn't 
feel  pressured  to  do  anything  difficult.  The  operation  of 
the  device  was  simple  to  learn. 

8.  No, the  operation  was  fairly  simple.  There  wasn't  that 
much  training  needed. 

9.  When  I  was  active  duty  I  worked  for  the  3246  Test  Wing  at 
Eglin  AFB  for  a  period  of  time.  So,  I  have  experience  with 
testing,  and  I  thought  the  test  was  conducted  fairly  well. 

11.  The  briefings  were  clear.  I  understood  what  was  expected 
of  me. 

14.  None  that  I  could  think  of.  It  was  very  easy  to  use. 

24.  No, it  all  seemed  fine. 

25.  Maybe  the  display  could  have  a  battery  monitor  so  you 
could  tell  when  you  were  getting  low  on  power. 


26.  No. 

27.  No,  it  was  fairly  clear  what  was  expected. 

28.  No. 

29.  It  seemed  satisfactory  to  me. 

30.  No. 

31.  No,  it  was  a  good  format  that  you  presented. 

************************************************************ 

Question  7:  Do  you  have  any  other  questions  or  comments? 

1.  This  would  benefit  traditional  guardsmen  quite  a  bit. 
Normally  the  technicians  who  work  here  full  time  walk  us 
through  the  procedure.  We  rarely  review  the  T.O.  It  is  too 
difficult  to  use.  Your  device  would  place  us  in  more  of  a 
control  position. 


2.  No. 

3.  Some  newer  troops  may  not  know  where  all  the  switches 
are.  It  might  help  them  if  you  provided  a  pictorial  of  what 
the  switches  are. 

4.  In  using  the  fault  locator  switch  in  your  test,  it  was 
nice  to  turn  the  fault  locator  switch  and  look  at  the  HMDD 
screen  without  having  to  turn  all  the  way  round  back  to 
check  the  T.O.  data.  The  information  was  right  there.  The 
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pistol  grip  was  easy  to  use  because  you  didn't  have  to  look 
at  it  to  push  the  correct  button.  You  could  do  it  by  feel. 


8.  No. 

9.  On  postflight  and  preflight  inspections  you  have  an 
intake  and  exhaust  portion.  It  would  be  difficult  to 
perform  these  inspections  while  wearing  the  HMDD.  On  the 
engine  exhaust  post flight  inspection  you  are  laying  down 
inside  the  engine  to  check  the  mixture  nozzle,  temperature 
nozzle,  and  aft  blades.  You  would  have  a  hard  time  insuring 
the  HMDD  components  wouldn't  snag  any  of  the  engine  parts. 

On  the  F-16,  there  is  more  room.  You  can  do  the  Inspection 
on  your  hands  and  knees.  The  intake  inspection  is  the  same 
way.  You  have  to  crawl  into  the  engine  to  inspect  the  air 
temperature  probe,  stater  blades,  variable  inlet  guide 
vanes,  oil  struts,  compressor  blades,  seal.  Generally,  we 
would  not  even  take  our  job  guide  with  us  when  inspecting 
the  inlet.  In  fact,  we  had  a  policy  to  take  off  our  boots 
before  inspect  the  inlet.  So  if  you  were  using  the  HMDD  you 
would  have  to  remove  it  for  the  inspection  and  put  it  back 
on  afterwards.  That  is  allot  of  extra  work  plus  there  is 
an  increased  chance  of  damaging  the  system  in  the  process. 


11.  It  would  have  been  helpful  to  have  drawings  of  what  to 
inspect.  What  are  the  items  being  asked  to  be  inspect. 

14.  You  may  look  at  using  a  heads  up  display.  You  could  be 
able 

to  see  through  the  information.  As  long  as  the  device  is 
durable  and  more  lightweight  it  would  be  useful  in  viewing 
T.O.  data.  The  higher  complexity  the  task  the  more  likely 
that  this  device  would  be  useful. 


24.  No. 

25.  You  might  look  into  putting  a  pause  function  in  the 
system  so  you  could  extend  the  battery  life  when  you  are  not 
reading  the  screen. 

26.  You  may  consider  using  a  quick  disconnect  to  the  vest 
and  use  aircraft  power  for  running  the  device.  We  use  a 
similar  connection  when  operating  the  doppler  ground  speed 
box.  We  plug  it  into  the  receiver /transmitter  and  run  the 
cable  up  to  the  cockpit.  The  problem  with  batteries  is  that 
they  can  go  out  at  an  inopportune  time.  It  seems  to  me  that 
batteries  are  too  unreliable  to  be  the  sole  source  of  power. 

27.  I  may  have  done  better  with  the  screen  placed  over  my 
non-dominant  eye. 
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28.  No. 


29.  Just  for  my  own  personal  use,  I  doii't  think  it  would 
work  very  well  for  me.  I  would  prefer  the  other  computer  so 
that  I  don't  have  anything  blocking  my  vision. 

30.  I  am  concerned  on  how  long  the  batteries  last.  Also  I 
am  not  sure  how  well  the  system  would  work  in  adverse 
weather.  If  it's  cold  out,  the  screen  may  tend  to  fog  up 
from  your  breath. 

31.  I  don't  think  the  device  would  work  with  chemical 
warfare  gear  on.  The  main  problem  would  be  the  vest.  You 
would  not  have  enough  room  to  place  the  eyepiece  under  the 
plexiglass  shield. 


141 


Figure  20.  Readability  of  Information 


Figure  22.  Display  Brightness 
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Figure  24.  Computer  Response  Time 
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Figure  27.  Eye  Strain  from  HMDD  Use 
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Figure  28.  Blurring  from  HMDD  Use 


Figure  29.  Difficulty  in  Focusing  from  HMDD  Use 


Figure  30.  Presence  of  After  Images  from  HMDD  Use 
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